Word Bocce

played on a court of 40,000 words

Raking the court…Loading the words (a few MB, once).

How to play

  1. Your red ball starts on one word. The yellow jack is another word. Roll your ball as close to the jack as you can.
  2. Tap words to add (+) or subtract (−) their meaning, up to three, then tap Throw.
    hat + foot − head → shoe
  3. Closeness is a ranking. 1st means the jack is the nearest word to your ball, a bacio (a kiss). You get four balls; your best counts.

Why does adding words work?

The game stores each word as a long list of numbers: its position on a map of meaning. Words with similar meanings get similar lists. A throw adds and subtracts those lists, so the ball moves through meaning.

You can choose which map the game uses: .

What does this have to do with chatbots?

Chatbots also store meanings as lists of numbers, but they read tokens (often pieces of words) rather than whole words, and they reshape the numbers using the words around them. The shows how, step by step, with a real tokenizer.

Why don't throws always make sense?

After any throw, tap Why? for a reading of what each word did. It's a best guess: .

Versus rules

Real bocce rules: whoever is farther from the jack throws next. When both sides are out of balls, the closer side scores a point for each of its balls that beats the other side's best.

The maths, in slides

Three maps of meaning

Every word in the game has a position on a map of meaning: a long list of numbers. Words with nearby positions count as close. You can choose where those positions come from.

Common sense

Built from how words are used in text, plus ConceptNet, a large collection of everyday facts that people have written down: "a hen lays eggs", "bacon is a kind of meat", "you wear a shoe on your foot". Throws mostly work the way you'd expect. It uses about 21,000 everyday words, and a court is only dealt if its best throw can be explained word by word.

Raw text

Learned only from which words appear together in billions of words of Wikipedia and news. This is the classic "word embedding" (GloVe) behind the famous king − man + woman ≈ queen. Nobody told it any facts, so it can surprise you: "ham" sits among football clubs (West Ham), and "eggs" sits nearer "meat" than "bacon", because that's how the news writes about them. Try it if you're curious how a machine that only reads text arranges words. ()

AI tokens

The table of numbers inside a real language model: GPT-2 (OpenAI, 2019), an early ancestor of today's chatbots. Its entries are about 50,000 tokens, not words. “␣shoe” (with the space in front, as in the middle of a sentence), “Shoe” and “␣shoes” are separate entries, and so are pieces of words, so a ball often lands next to a near-copy of a word. Chatbots look up numbers like these for every token you type, before anything else happens. ()

Each has its own Daily. In an online room, the host's choice applies to everyone.

The Tokens tab shows how a chatbot gets from tokens to an answer. .

Why did that happen?

Why the game can't fully explain itself

The "Why?" notes are our best reading of the numbers, not the real reasons. A few things make this hard:

Modern AI systems, including chatbots, are built from learned numbers like these at a vastly larger scale. Working out what they represent and why they behave as they do is called interpretability. It's an open research problem: researchers can now find some meaningful features inside models, but nobody can fully explain a large one.

Further reading: Mapping the mind of a large language model (Anthropic, 2024) · Zoom In: an introduction to circuits (Olah et al., Distill, 2020).