ChatGPT wrote 50 science abstracts, and reviewers missed about a third

Here's the setup. Take 50 real abstracts from five high-impact medical journals. Give ChatGPT nothing but each paper's title and the journal's name, and ask it to write the abstract. Then show the results to blinded human reviewers, mixed in with the real ones, and see who can tell.

That's the study a group of researchers from Northwestern University and the University of Chicago posted to bioRxiv on December 27, 2022. Catherine Gao was the first author. The preprint hadn't been peer reviewed yet, but it got passed around fast, and it's easy to see why.

The numbers

The reviewers correctly identified 68% of the ChatGPT abstracts. Which means roughly one in three slipped past people who knew they were looking for fakes. They also flagged 14% of the genuine abstracts as machine-written, so their instincts misfired in both directions.

The software did better. An AI output detector gave the generated abstracts a median score of 99.98% and the real ones 0.02%, where higher means more likely machine-written. A plagiarism checker, on the other hand, rated the generated abstracts 100% original, since they weren't copied from anywhere. They were invented.

Two more findings stuck with me. Only 8% of the generated abstracts followed the journal's specific formatting rules, so the model knew roughly what an abstract looks like but not each journal's house style. And the patient numbers were made up. The generated abstracts used cohort sizes in a believable range, similar to the real papers. The specific figures were fabricated.

Why that last part matters to you

You're probably not faking medical research. But you might ask a chatbot to draft a report, a proposal, a cover letter, or a product page. If your prompt doesn't include the real numbers, the model will still write something with numbers in it, because that's what those documents look like. The study shows exactly how convincing those invented figures can be. Trained reviewers couldn't reliably spot them.

This is my strongest opinion about AI writing: the model should do the wording, and you should supply the facts. Every time.

Before:
Write a one-paragraph summary of our school fundraiser
for the newsletter.

That prompt is an invitation to invent. You'll get a paragraph about "record turnout" and "over $5,000 raised" whether or not either is true.