AlphaCode made the cover of Science by writing lots of programs and keeping a few

A coding contest problem is a little story with a hard ending. Some setup about a grid, or a queue of people, or numbers on a board, then a strict input format, strict time limits, and hidden tests that don't care how clever your idea sounded. On December 8, 2022, a paper about a machine that tries these problems ran in Science, on the cover. It was DeepMind's AlphaCode.

ChatGPT was eight days old, and its chat window handed you one answer per question. AlphaCode came at code from a different direction, and the difference is still useful.

What DeepMind actually claimed

DeepMind had introduced AlphaCode back in February 2022, and updated its blog post after the Science publication. The headline number was modest on purpose. Tested by simulating participation in 10 recent contests on Codeforces, a popular competitive programming site, AlphaCode reached an estimated rank within the top 54% of participants. DeepMind described that as roughly the level of a median competitor.

Median doesn't sound like much. But the problems require reading a long plain-English description and turning it into working code, and that's what made it notable.

The method is the interesting bit

AlphaCode was pre-trained on public code from GitHub, then fine-tuned on a dataset of competitive programming problems. So far, normal.

Then it did something that sounds almost brute force. For each problem it generated a very large number of candidate programs in C++ and Python. It filtered them, clustered similar ones together, and reranked what was left, until it had narrowed the pile down to 10 candidate solutions to submit.

I find this more instructive than the score. The system didn't trust its first try. It didn't trust any single try. It assumed most attempts would be wrong and built a process for picking the good ones out.

What that means for an ordinary chat

You can't run thousands of samples in a chat window, and you don't need to. The lesson scales down nicely: asking for one answer gets you whatever the model reached for first. Asking for a few different ones, then choosing, gets you a better floor.

People skip this because it feels like extra work. Reading three short options and picking one takes less time than arguing a mediocre draft into shape over six follow-up messages. And comparing them teaches you what you actually wanted, which you often don't know until you see two side by side.