Chatbots & language models

You ask a question, and seconds later there is an answer that sounds like a human wrote it. For many people chatbots have become as everyday as the search engine. That makes it all the more interesting to ask what is behind them. The short answer: a lot of mathematics and a surprisingly simple core idea.

Text becomes numbers

A computer does not know letters, only numbers. Long before AI, every character was therefore given a fixed number: in the ASCII code A is 65 and B is 66. In memory such a number is stored as a pattern of 8 bits, that is one byte. Language models go one step further and break text into small building blocks called tokens. These can be whole words or parts of words, and every token gets a number. To the model, your entire chat is one long chain of numbers.

Predicting the next word

At its core a language model plays a guessing game: which token comes next? After the sentence start 'The sky is', the word 'blue' is very likely and 'banana' very unlikely. During training the model read huge amounts of text and learned to compute a probability for every possible continuation. An answer is built by choosing token after token, and every chosen token feeds back into the prediction for the next one. How much text it can keep in view while doing so is limited: that window is called the context and is measured in tokens. Once it is full, the beginning of the conversation drops out.

Why this works so well

Guessing words sounds trivial, but to predict the next word really well the model has to capture an astonishing amount: grammar, context, factual knowledge, even the tone of a conversation. To correctly continue 'The capital of France is', the answer has to be encoded in its patterns. From billions of such patterns emerges a system that can translate, summarise and explain. Not because it understands the task like a human, but because good prediction forces exactly these abilities.

What a language model is not

Two misconceptions persist. First: a language model is not a database. It does not look answers up, it generates them fresh every time from probabilities. That is why it can be wrong and still sound convincing. Second: it has no consciousness and no intentions. When a chatbot writes 'I think', that is a learned language pattern, not a sign of an inner life. Keep both in mind and you can use chatbots with confidence: as a powerful tool, not as an all knowing being.