Integrating OpenAI's GPT-4 into Games

Here is Chat-GPT's summary of the article I painstakingly wrote. If instead you want to read my meandering prose, you can skip to the original start of the article below.

Ever since I got access to OpenAI's APIs, I've been searching for ways to integrate it into my website. I used it to develop content, help me program games, create a webtool for asking AI, and now, I'm going to start integrating it into Games on The Internet.

The first game using this is Debate an AI. You can face off with one of OpenAI's large language models (LLMs) to see who can come up with the best argument for the prompt. Are you smarter than an AI? I decided on this game for a few reasons:

• Game has a simple structure, but utilizes the unique capability of GPT-4 to understand complex information and make subjective decisions
• Will be interesting to see how each of OpenAI's LLMs stack up against the average user of The Internet
• Debate topics are thought-provoking and are often important issues that we should be thinking deeply about
• Having an AI be able to judge the validity of arguments has intriguing implications/applications for fact-checking, speech & debate, and comment section moderation

Ultimately, I created a game with two modes and three difficulties, a detail page for each game so you can see the debate history, a detail page for each debate prompt so you can see all arguments related to each debate prompt, and a list page of all of the debate prompts so you can browse arguments by topic. The overarching structure of the game was actually the hardest, but I'll focus on the use of OpenAI's API in my reflections since that's more interesting.

After building this game, here are some things I learned:

• GPT-4 is pretty incredible at judging the validity of arguments. To get the winner of each debate, I provide GPT-4 with the arguments and tell it to decide which is better and provide its reasoning - in my testing, there weren't any decisions I disagreed with. Asking it to also provide its reasoning probably adds to its accuracy
• Using a different model for each difficulty worked. Easy = GPT-3, Medium = GPT-3.5, Hard = GPT-4. At the start of this experiment, I wasn't sure if this was going to work - would there not be enough of a noticeable differentiation between the performance of the models? would GPT-3 even be capable of reliably providing some form of argument capable of winning? Thankfully, they operated just as expected, giving a good progression in difficulty. It's unclear if using this technique for more complex games would work or if just using GPT-4 and instructing it to behave in a different ways depending on difficulty chosen would be better.
• GPT-4 is much more capable than GPT-3.5 at following instructions and providing consistent output. The most challenging aspect of this project was also probably the least necessary.. I was worried that providing GPT-4 might be biased if they knew which argument was made by the human or if one argument was always presented first. In retrospect, these probably weren't huge issues. But alas, I implemented a coin flip mechanism to decide which argument was presented first and anonymized the input to GPT-4. Translating the anonymized output from GPT-4 back into one the player could understand was surprisingly difficult. GPT-3.5 was producing inconsistent results in deanonymizing the output, so I had to upgrade to using GPT-4 and it worked easily. I probably could have solved the issue by just providing more than one example of a correctly translated text, but I didn't want to increase token usage so much.

Anyway, those are my longwinded thoughts on using GPT-4 in a game. To make them more readable, I've asked Chat-GPT to summarize them with some engaging bullet points. It's actually better than what I wrote, so I'm starting the article with it.