Meta trained a model on 2,000 tasks so it would follow plain instructions

Three days before Christmas 2022, while most of the internet was still arguing about ChatGPT, a group of Meta researchers posted a paper with a title only a researcher could love: "OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization."

Strip away the title and the idea is simple. Take a big language model and teach it, on purpose, to do what it's told.

What went up that day

The paper, by Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov and colleagues, was submitted to arXiv on December 22, 2022. According to its abstract, the team built a benchmark called OPT-IML Bench with 2,000 language tasks, drawn from 8 existing benchmarks and grouped into categories. They worked out their training choices on a 30-billion-parameter OPT model first, then trained OPT-IML at two sizes, 30B and 175B.

They tested the results on four outside benchmarks (PromptSource, FLAN, Super-NaturalInstructions and UnifiedSKG). The abstract also says they released OPT-IML at both scales, along with the benchmark's evaluation framework.

That's the whole news item. No launch event, no app. But the paper is a good window into a shift that was happening across the field in late 2022. Models were being trained on huge piles of examples written as instructions, so that a plain request like "summarize this for a ten-year-old" would just work.

Why that changes how you should prompt

A lot of early prompt advice was built for models that weren't instruction-tuned. You had to trick them. Start a fake document, write the first line of the answer yourself, hope it continued in the right direction. Some of those habits are still floating around, and I think most of them are dead weight now.

What instruction tuning rewards is the opposite of tricks. Clear request, real context, and then (this is the part people skip) actually reading the answer to see what the model thought you meant.

Because when an instruction-tuned model gets it wrong, it usually isn't ignoring you. It's following a different reading of your words. The fix lives in your sentence, not in a magic phrase.

Read the answer back

Here's the habit. When a response misses, don't just say "no, try again." Hold the answer up against your prompt and ask: what reading of my words would produce this?