Hello, Model

The Whole Trick

Strip away the acronyms and the product names and the breathless headlines. A language model does exactly one thing.

You send it text. It sends text back.

That is the whole trick. When you chat with an AI, an app wraps what you typed with some standing instructions and sends the bundle to a model. The model writes a reply. The app shows it to you. The model is not browsing the web. It is not consulting a database. It does not remember your last conversation. If any of that happens, something around the model did it. This book is about that something.

A useful mental picture: the model is a very well-read colleague behind a mail slot. You pass a note in. A note comes back. Nothing else goes in or out.

Everything else in this book is people arranging things around that mail slot.

Four Words You Cannot Avoid

You need four words. Everyone in this space uses them constantly and none of them are complicated.

A prompt is the text you send. That is the entire definition. The standing instructions an app adds around your message are called a system prompt. It is still just text.

A token is the small chunk a model actually reads and writes. A common word like "the" is usually one token. A long word like "orchestration" gets broken into a few pieces. One hundred tokens is roughly seventy five words. Model pricing and model limits are counted in tokens. That is the only reason the word comes up so often.

A context window is how much the model can consider in a single call. Everything draws from the same budget. Your message. The instructions. Any pasted documents. Even the reply the model is writing. The big models today can hold hundreds of pages at once. That sounds like infinity until the first real project fills it. When the budget is full, something has to be left out. Deciding what makes the cut turns out to be one of the biggest jobs in this field.

A model is the trained thing itself. Some models are smarter. Some are faster. Some are cheaper. Picking one is closer to picking a rental car than joining a religion, whatever the internet suggests.

Those four words are most of the vocabulary gate at the front of this space.

The Goldfish Rule

Here is the fact that surprises almost everyone.

The model remembers nothing between calls.

Each call starts completely blank. When a chat app seems to remember your name from twenty messages ago, it is because the app quietly resends the entire conversation with every new message. The memory lives in the app. Not the model.

This one fact quietly explains half of AI engineering. Most of the systems this book describes exist to select what the model sees on each call. They exist because the model keeps nothing on its own.

It Writes. It Does Not Know.

One more honest thing and then you have the full picture.

A model does not look up answers. It writes the most plausible next text one token at a time. It draws on everything it absorbed during training. It is right remarkably often because it absorbed an enormous amount of writing that was right. And sometimes it produces something confident and fluent and wrong. People call that a hallucination.

This is not a defect you fix by scolding the model. It is a property you design around. You give the model the facts it needs inside the context window. You check its work on the way out. Those two moves have their own chapters later. For now it is enough to know why they exist.

What You Are Allowed To Ignore

The space looks bigger than it is. Most of it can wait.

You do not need the math. You do not need to know how models are trained. You do not need a GPU. You do not need to understand the word transformer beyond nodding at it politely. People who build serious AI systems treat the model as a component. You can use a database for a decade without once writing one.

Remember four things and you know enough for the next chapter. Text in. Text out. Four words. The goldfish rule. It writes. It does not know.

Next the model gets hands.