Day two of ChatGPT, a pirate debugger and a bluffing Stratego bot

On Thursday, December 1, 2022, a lot of people were asking ChatGPT to fix their code. Some of them asked it to do it as a pirate. The Verge's James Vincent quoted one reply that opened with "Arr, ye scurvy landlubber!" before explaining the problem with a loop condition.

That review went up at 6:22 PM UTC, roughly a day after launch, and it's still one of the fairest snapshots of what the thing was like.

What The Verge saw

Vincent's piece called ChatGPT fantastically talented and also prone to producing cogent waffle and misinformation, which is a pretty good two-line summary of the next few years. He noted it was adapted from OpenAI's GPT-3.5 model and trained to give more conversational answers. People were using it to explain science, write basic academic essays, and produce uneven sitcom scripts that mixed actors from different shows.

The failures were just as visible. Computational biologist Carl Bergstrom asked it to write a Wikipedia entry about his own life, and it did, smoothly, with several false biographical details. The safety rules were also easy to get around: ask for something dangerous directly and it refused, but frame it as a character in a film and it would sometimes answer anyway.

Meanwhile, a bot that bluffs

The same day, DeepMind announced DeepNash, an agent that learned Stratego from scratch by playing against itself, with the work published in Science. Stratego hides your opponent's pieces from you, so you have to bluff and read bluffs. DeepMind reported that DeepNash won over 97% of matches against top Stratego bots and 84% against top human experts on the Gravon platform, reaching an all-time top-three ranking there. In one game it chased an opponent's 8 with a lowly Scout, so the opponent assumed it was a strong piece.

Two very different systems in one news day. One was built to win with hidden information. The other was built to talk, and it sounded just as sure when it had no idea.

The pirate lesson

I like the pirate example more than most people would, because it shows the cheapest and most reliable thing you can steer: voice. Tell the model who's listening and the tone shifts instantly. What it doesn't do is make the facts better. The pirate still has to be right about the loop.

So use the reader trick for what it's good at. A parent, a new hire, a nine-year-old, a lawyer: naming the reader changes word choice, length and how much is assumed. It's the closest thing to a free upgrade in prompting. Just don't confuse a better voice with a more correct answer.