What Is a Language Model? A Plain-English Guide

A language model is a program that has learned to guess which text comes next. You hand it a stretch of writing, and it returns the most likely continuation based on patterns pulled from an enormous pile of documents. That one trick, repeated token by token, produces essays, code, translations and chat replies. Knowing how it works explains both why it feels clever and why it invents things.

Next-token prediction, explained without maths

The model does not store sentences and look them up. During training it reads billions of words and adjusts millions or billions of internal numbers, called parameters, so that its guesses match the text it saw. Sentences are first chopped into tokens, which are roughly word-sized fragments. For each position the model produces a probability for every token it knows, picks one, then repeats the process with that token added. Ask it to finish The cat sat on the and the highest scores land on mat or floor.

Nothing in that loop checks whether the sentence is true. The model is a statistical mimic with an extraordinary memory for shapes. When its continuation is accurate, it is because accurate text was common in its training material. When the continuation is wrong, the sentence usually still reads smoothly, and that gap between fluency and truth is the source of most trouble.

Where the knowledge comes from

Training usually draws on web crawls, digitised books, code repositories, reference works and forum threads. That material is collected up to a cutoff date, so a model trained in 2024 has no knowledge of events after its training ended unless a search tool is wired in at the front end. It does not update itself while you talk to it.

Because the mix is broad rather than even, coverage is uneven. Popular languages, mainstream science and well-documented software appear many times and are reproduced fairly reliably. Niche dialects, recent local news, private company records and anything behind a login wall are thin or absent. A model that writes confidently about a small town bus timetable is usually filling gaps with plausible shapes rather than recalling a real schedule.

Why the answers sound so assured

Fluent grammar comes free with the training objective. There is no separate confidence meter inside the system. The same process that produces a correct paragraph produces a fabricated citation, a made-up statute number or a non-existent study. This is called hallucination, and it cannot be patched out completely, because the model has no built-in way to separate remembered fact from well-formed invention.

Models tuned with feedback from human raters also tend to sound helpful and agreeable. That tone can hide uncertainty. Ask whether a claim is correct and a chat model may agree with you rather than push back, because agreement was rewarded during tuning. Treat a confident voice as style, not as evidence.

What a language model cannot do on its own

It has no persistent memory of you between sessions unless the product stores a profile. It cannot see, hear or browse unless it is given tools that do those things. It cannot perform arithmetic by instinct; long multiplication is pattern matching and slips often. It has no access to your files, accounts or inbox unless you connect them.

Most importantly, it does not reliably know when it does not know. Asking a model to flag its own uncertainty helps a little, but the flag is generated the same way the answer was, so it can be wrong in the same direction.

Words worth knowing

• Token - A chunk of text, often part of a word, that the model treats as a single unit when reading and writing.
• Parameters - The internal numbers adjusted during training. More of them mean more capacity, not more accuracy.
• Training data - The documents the model learned from before release. It sets both the strengths and the blind spots.
• Context window - How much text the model can hold in view at once, measured in tokens.
• Temperature - A setting that controls randomness. Low values give predictable output, high values give varied and less reliable output.
• Inference - Running a trained model to produce an answer. This is when you use it, not when it learned.

How to use one without being misled

Read model output as a draft from a fast, widely read assistant who never checks anything and cannot be embarrassed. Verify names, numbers, quotes, dates and any legal or medical claim against a source you trust. Keep the original documents in front of you and ask the model to work from those rather than from memory.

For work where being wrong is cheap, such as brainstorming titles or tightening a paragraph, the guesswork is harmless. For work where being wrong is expensive, the model should be the first draft and a human or a primary source should have the last word.