A.L.I.C.E. and AIML: the chatbot built from 41,000 handwritten answers

Somebody types "Tell me about Socrates." Somebody else types "Describe Socrates." A third person asks "Do you know Socrates?" To a person those are one question. To a chatbot in the late 1990s they were three separate strings, and unless a human had written a rule for each, at least one of those people got a dumb answer.

That problem shaped A.L.I.C.E., the chatbot Richard Wallace started in 1995, and the way he solved it is still the best prompting habit I know.

A brain made of index cards

In his paper The Anatomy of A.L.I.C.E., Wallace says the first edition was implemented in 1995 in SETL, a little-known language based on set theory. It drew few contributors until it moved to Java in 1998. The next version, Program B, launched in 1999, had more than 300 contributing developers, and switched AIML to a fully XML-compliant grammar.

AIML stands for Artificial Intelligence Markup Language. The unit of knowledge is a category: a pattern (what the user might say) and a template (what the bot says back). Wallace described the A.L.I.C.E. brain as roughly 41,000 of these. Patterns could include wildcards, so one rule might catch many sentences, and the software stored them all in a tree it called the Graphmaster for fast matching.

He didn't hide the family tree. The paper says some people saw A.L.I.C.E. as a simple extension of ELIZA, and Wallace's answer was scale: more than 40,000 categories against roughly 200 in the original ELIZA. Wikipedia also notes the bot was inspired by Weizenbaum's program. It won the Loebner Prize, an annual Turing Test contest, in 2000 and 2001 according to Wallace's paper, and again in 2004 according to Wikipedia.

Targeting: reading the failures

This is the part I'd put on a poster. Wallace described two ways to write content. The first was anticipatory: guess every way someone might phrase a question. There was even a "Knowledge Wizard" that generated variations like the Socrates ones above. His verdict: "humans are notoriously bad at predicting which patterns will be activated."

The second way looks backward. The botmaster reads the logged conversations, notes the incorrect replies, and writes new categories for those inputs. Wallace leaned on Zipf's law here: if one person says something, there's a nonzero chance another person will say the same thing later, so you fix the most common misses first. He called the process Targeting.