By skyforbes 
An explanation of ML science for the casual prompter to ensure results of a higher quality for work and AI automations
Introduction
Have you ever noticed that during long conversations with your favorite AI agent, it forgets some of the text you wrote or files you attached at the start? You’re not alone and this is totally “normal”.
This forgetfulnesss is not an AI-personality trait or a personal motive of an AI-agent that dislikes you… no, it’s because of something called context windows.
In this article, I’ll transalte some of the Machine Learning science and explain why.
What is a context window?
When you Chat with ChatGPT, Claude, Gemini or other models, everything that you write including the writings provided through uploaded media is turned into small pieces called tokens.
A context window is the limit of how many tokens the model can remember at once. The larger the context window, the more the model can remember. You can think of it like our short-term memory.
Early versions of ChatGPT could handle about 4,000 tokens (roughly 3,000 words)
The latest models can handle hundreds of thousands of tokens.
Therefore, AI agents become more capable with time, however, this is always in relation to the input we give. Imagine that one day an AI model is connected to a video live-feed from your smart glasses, where all objects and images you see IRL are translated into words that represent the context for the AI. You just walked past a trashbin without noticing and now, a minute later you’re asking the AI model if there’s a bin nearby you. Well in that case we might require a context window in the millions to still see/remember the data from 1 minute ago. (I’m not particularly excited about an AI future where we don’t use our eyes anymore, but for the sake of the example this should suffice).
To conlude this section, this example shows why the “issue” of your AI agent being forgetful might never actually be solved. Even more important to gain an understanding now, so you can use it better.
How Models Use Context
Let’s get back to the actual context window. Well, inside it, the AI doesn’t actually store facts, like the
How you can use this knowledge
Summarize long chats: Take a break after long exchanges to remind the model about what is important with regards to your chat history.
Keep prompts focused: Avoid unnecessary details that pollute the context window. This way you actually keep some of your brain cells, those that are needed to express yourself concisely 😉
Split up long documents: To avoid that a long document is only rememberd partly, split it up and summarize it as you go. While summarizing double check that certain important details are not lost in abstraction.
Conclusion
Understanding context windows will help you empathize with your AI model. In addition, you will be able to mitigate producing unclear or incomplete outputs. This directly influences the quality of your work. And while you can still double check the outcome of a manual prompt, you might run into trouble when you’re working with AI automations.
When there is no instance designated to check the accuracy of the models output and you “overwhelm” it beyond it’s context window then your automation is designed to fail in terms of quality.
So if you want smarter results keep context windows in ya context window! (ba-dum-tss).