Training Knowledge vs. Conversation Context

Hello. In the previous lesson, you saw that an LLM produces text one token at a time, using the context currently available to estimate plausible next tokens. That raises a crucial question: where does the model get the knowledge and skills that make those predictions plausible in the first place?

This lesson separates two sources that work together in every response:

1. Information and patterns learned during training, encoded in the model’s weights.
2. Information supplied at runtime, such as your prompt, chat history, instructions, examples, and attached documents.

Making this distinction accurately will help you prompt better, avoid overclaiming what an AI “knows,” and choose sensible ways to handle new or organization-specific information.

---

Two different mechanisms, not one kind of “memory”

A useful conceptual model has two parts:

• Model weights are the many numerical parameters established during training. They influence how the model interprets language and which continuations it considers likely.
• Context is the text and other information made available for the current request. It gives the model the immediate situation it should respond to.

We can express the next-token prediction from the previous lesson as:

$$P(\text{next token}\mid \text{context},\text{weights})$$

The current context changes whenever you send a new message or attach a new document. The model’s weights normally do not change when you chat with it.

That distinction is more important than it may initially seem. Consider these statements:

• “The model can write a professional email.”
  This mostly draws on writing patterns learned during training.

• “The client’s meeting was moved to Thursday at 2 p.m.”
  This fact needs to be provided in the current conversation, retrieved from a connected source, or otherwise made available at runtime if it is recent or private.

• “Use this company’s approved email sign-off.”
  The sign-off can be supplied as an instruction or example in the context. It does not need to become part of the model’s weights for the model to use it in this response.

A model’s answer is therefore not simply retrieved from a fixed internal library, nor is it created solely from your prompt. Its trained parameters supply broad learned capabilities; the current context steers those capabilities toward a particular response.

---

What training changes

During pretraining, a model is exposed to very large collections of text and repeatedly trained to predict the next token. The system compares its prediction with the actual next token in training material and adjusts its internal numerical parameters. Repeating that process at enormous scale produces weights that encode useful statistical patterns.

Those patterns can include:

• grammar, vocabulary, and writing conventions;
• relationships among concepts commonly represented in training material;
• programming patterns and common document structures;
• broad tendencies about how explanations, summaries, translations, or dialogue are written.

Later training stages can further shape how the model behaves as an assistant. For example, training on high-quality instruction-and-response examples can make a model more likely to follow requests, use a clear format, or refuse unsafe requests.

This does not mean the model has a neat internal folder containing every training document. Its weights are not a searchable archive with reliable labels such as “source,” “date,” and “author.” The model may generate a useful explanation based on patterns it learned, but it generally cannot prove where a particular claim came from or reliably identify whether a specific webpage was in its training data.

A practical implication follows: a model has a knowledge cutoff. Events, policies, product changes, or research published after its training cannot be assumed to be reflected in its learned weights. Even information that was available before training may be incomplete, misrepresented, or recalled inaccurately in a particular answer.

So “learned during training” should never be interpreted as “guaranteed true, current, or complete.”

---

What a current conversation supplies

At response time, an application constructs a request for the model. This is called inference: the model runs using its already-trained weights rather than being trained again.

The supplied context may include:

• system or developer instructions that set behavior and constraints;
• your current message;
• earlier messages that the application includes as conversation history;
• an uploaded document or excerpts from it;
• examples of desired inputs and outputs;
• information returned by an external search, database, or tool;
• the tokens the model has already generated in its current response.

Unlike training, this information is temporary. It is available because it has been included in the current request, not because it has been integrated into the model’s weights.

Imagine you upload the current version of an employee handbook and ask:

Based only on the attached handbook, explain the annual-leave approval process for a new employee in five bullets.

The model’s trained weights provide its general ability to read, interpret, and write a five-bullet explanation. The handbook provides the specific, current policy information. If the handbook changes tomorrow, you can provide the revised version in a new request; no retraining is required.

The current context has limits. As you learned previously, it must fit inside a finite context window, along with the output the model is generating. A long conversation may be shortened, summarized, or partially omitted by an application. Therefore, a model does not automatically retain every past chat forever.

When a chat product appears to “remember” something, one of several things may be happening:

• Earlier messages are being sent again as part of the current context.
• The application saved a summary, preference, or instruction and injected it into the new request.
• A Project or similar workspace is making designated files and instructions available.

In each case, the information is best understood as runtime context unless the model itself has undergone a separate training process.

---

The key test: did the weights change?

When in doubt, use this diagnostic:

Situation	What changed?	Category
You paste a new company policy into the chat	The current request contains new text	Supplied context
You attach a spreadsheet and ask for a summary	The application makes file content or relevant excerpts available	Supplied context
You give three examples of the format you want	The examples guide this response	Supplied context
An application retrieves current database records before asking the model to answer	Retrieved material is added at runtime	Supplied context
A model developer runs additional optimization on a training dataset	Internal parameters are adjusted	Training or fine-tuning
You have a normal multi-turn conversation	The model generates responses; its deployed weights stay fixed	Supplied context

The decisive question is:

Was an offline training process used to update the model’s weights?

If yes, the information or behavior was shaped through training, perhaps through fine-tuning. If the information was simply placed into the request, chat history, workspace, or retrieved documents, it is context.

Fine-tuning versus giving an example

Fine-tuning is a form of additional training. A developer uses a specialized dataset and an optimization process to alter model weights. It can shape a model’s behavior or performance for a recurring domain-specific task, but it requires deliberate technical work, quality data, evaluation, and maintenance.

By contrast, placing examples in a prompt changes only the immediate context. This is called few-shot prompting or in-context learning.

In the image, the model sees:

1. A task instruction: translate English to French.
2. Several example translations.
3. A new item to translate.

The examples make a particular continuation more likely: a French translation in the same compact format. They do not retrain the model or permanently teach it a new language fact. In the next module, you will use this principle deliberately when designing few-shot prompts.

---

Both sources are necessary in a professional workflow

It is tempting to ask whether an answer came from training or from context. Usually, the right answer is both, but they make different contributions.

Suppose a support assistant receives this context:

Product version: 4.2
Known issue: exports fail when file names contain an ampersand.
Approved workaround: rename the file without special characters, then export again.
Reply to the customer in a calm, concise tone.

The model uses its learned capabilities to understand the request, recognize the style of a support response, and compose fluent text. It uses the current context to include the specific version, issue, and approved workaround.

If it were not given the product details, a generally capable model might invent a plausible troubleshooting step. If it had the product details but no trained language ability, it could not turn them into a useful response. Reliable work combines appropriate contextual evidence with the model’s general learned capabilities.

This also clarifies the role of retrieval-augmented generation, usually shortened to RAG. In a RAG workflow, an application finds relevant information in a knowledge base and places it into the model’s current context before generation. The model is not automatically “learning” the retrieved document in the training sense. It is using that document for the present request.

For current, private, or frequently changing information, providing or retrieving source material is usually more appropriate than relying on what a model may have learned in training.

---

Language to use precisely

Avoid these misleading shortcuts:

Misleading statement	More accurate version
“The chatbot learned our policy when I uploaded it.”	“The chatbot used the uploaded policy as context for this request.”
“It remembers everything from our last chat.”	“It can use earlier messages if the application includes them in the current context.”
“I corrected it, so it will know that forever.”	“The correction can guide this conversation; persistent improvement requires a separate system or training process.”
“The model was trained on the internet, so it knows the latest news.”	“Its learned knowledge has a cutoff and may be incomplete; current facts should be sourced.”
“Few examples retrain the model.”	“Few-shot examples steer the response without updating the model’s weights.”

One final nuance matters for responsible use: a provider may have policies about whether and how user interactions can be retained or later used to improve services. That is a separate data-governance question. It does not mean that an ordinary chat immediately updates the deployed model while it is responding to you. For professional work, never assume that typing a fact into a chat has permanently taught that fact to the model.

---

Key takeaways

A language model’s weights encode patterns acquired through pretraining and later training stages. Those weights support broad abilities such as interpreting language, writing, coding, and recognizing common patterns, but they are not a reliable, current, searchable archive of every training source.

The current conversation context consists of information supplied at runtime: instructions, prompts, chat history, examples, uploaded files, and retrieved sources. It can strongly guide a response without changing the model itself. Few-shot examples, file uploads, and RAG are all ways of supplying context; fine-tuning is different because it changes weights through additional training.

Next, you will use this distinction to judge whether a task is a good fit for generative AI, including when human review and stronger evidence are necessary.