What a context window actually holds

A context window is the text a model is allowed to use while it writes one reply. The count is in tokens, which are not the same thing as words. The system instruction, the earlier turns, any passages a search step attached, and the question you just typed all draw from the same budget.

When people say a model has a long context, they are talking about that budget. They are not talking about a filing cabinet that keeps every chat you have ever had.

What shares the budget

A product screen shows your message. The request behind it is larger. A hidden instruction tells the model how to behave. A retrieval step may paste three chunks from a handbook. The previous ten turns may still be attached. Your new sentence is the last piece, and it is often the smallest.

If the handbook chunks are long, they push the older turns out. If the hidden instruction is long, it leaves less room for the document. The window is one pile of text, and every extra paragraph is a paragraph something else cannot occupy.

Some products trim the oldest turns until the request fits. Some refuse and show an error. Some summarize the history and send the summary. The documentation for the product you are using is the place that says which of those it does. A limit printed on a marketing page is not a promise that every token in the window is treated with the same care.

The middle of a long paste

Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, and Liang described this in "Lost in the Middle: How Language Models Use Long Contexts." Relevant text placed at the beginning or the end of a long context was used more reliably than the same text placed in the middle.

That matches a failure that is easy to reproduce without a lab. Paste a long policy, ask about a clause that sits halfway through, and the reply talks about the opening definitions. Repeat the question with that clause copied under the question, and the reply changes. The model did not gain a new fact. You moved the fact to a place the request was more likely to use.

A 200-page PDF dropped into a chat box is the same shape of mistake. The upload may be truncated before the model sees it. If it is not truncated, the clause you care about can still sit in the weak part of the window. Name the page, or paste the clause.

What "memory" usually is

A toggle labeled memory is a store outside the window. The product writes a short note, then inserts that note into a later request. If the note was never written, or was written wrong, the later request does not contain the original conversation. Clearing the store removes the note. It does not rewind the model.

A custom instruction is the same kind of object. It is text added to each request until you edit it. It consumes window space on every call.

How to spend the budget

Put the question and the passage that answers it next to each other. If you retrieved three chunks, the chunk that contains the answer should be easy to see, which is the whole point of retrieval before the model writes. Cut boilerplate from the source. A header repeated on every page of a PDF costs tokens and adds nothing.

Ask for a short quote from the source when you need to check the reply. A quote you can find in the passage is evidence the passage was in the request. A confident paragraph with no overlap with the passage is a reason to look at what was actually sent.

If you are comparing two models, compare them on the same attached text. A model with a larger advertised window is not automatically the one that answered from the page you meant.

What to save when a reply ignores the page

Save the request, not only the chat bubble. The request is the instruction, the turns that were still attached, the passages, and the question. If the product will not show you that payload, you cannot tell a trimmed history from a model that skipped the middle.

When you test a long document, put a unique sentence at the start, another in the middle, and a third at the end. Ask for each sentence in three separate calls, with the same attachment. If the start and the end come back and the middle does not, you are seeing the long-context pattern the Liu paper described, on your own file. If none of them come back, the attachment was truncated or never sent. Those are different repairs.

A summary step is another way to spend the window. Summarize a section, check the summary against the section, and send the summary onward. You have now put a shorter text in the next request, and you have a place to see what was dropped. A summary that missed the number you care about will poison every later reply that trusts it. Read the summary once. That reading is cheaper than arguing with the last reply in the chain.