Glossary
Every term the notebook uses, in plain language, pointing to the note that explains it properly. A place to look something up, not to learn it.
- Context window
- The fixed amount of text a model can have in front of it at one time: your question, the conversation so far, any document you pasted, and the answer it is writing. When it is full, the oldest text drops out. It is a working space, not a memory.Explained properly in Note 09, Section 3.
- Model
- The trained system that turns text in into text out. It is not the same as the product you use it through: the chat window, the search, the memory features and the safety rules are wrapped around the model by whoever built the product.Explained properly in Note 12, Section 3.
- Token
- The unit a model actually reads and writes. Roughly a word or a piece of a word: "notebook" may be one token, "strawberry" may be several. Everything the model does, and everything it costs, is counted in tokens.Explained properly in Note 02, Section 1.
Open questions
Some are open in the field: nobody has a settled answer yet, and the notebook will say what is known and what is not. Others are queued for a note. When a note takes one on, the entry stays and gains a link. Add yours by replying on LinkedIn.
- Queued for Section 3Why do models read the middle of a long document worse than the start and the end?
- Queued for Sections 3 and 8How much of what feels like a model's personality comes from its training, and how much from the instructions the product puts in front of it?
- Open in the fieldIs there a reliable way to tell a model's training cutoff from the outside, without being told? Researchers have proposed tests; none is settled.
Where I was wrong
Typos are fixed quietly. A claim, a number or an explanation is never changed without leaving the original struck through in the note and a line here.
Nothing yet. When there is, it will be listed here with a link to the exact place.
How this notebook is made
The notes are written by me, in my own words, and every claim in them is checked by hand. The practicals are run by me, on my own accounts, and the results are recorded as they came out. I work with an AI assistant to draft, to draw in the notebook's line style, and to build these pages; that seems only right for a notebook about how AI works, and I would rather say so than let you wonder. The drawings are original. The learner is mine.
a notebook should say how it was made.