AI Model Lifecycle: Training, Validation, Deployment, and Inference

Hello again. In the previous lesson, we separated the roles that can coexist inside an AI product: explicit rules, predictive models, generative components, and foundation models. The practical implication was to examine an AI system component by component rather than accept “AI-powered” as an explanation.

This lesson adds a time dimension. A model is not simply “built” and then used forever. It is trained on past data, checked before release, deployed into an operational system, and run repeatedly on new inputs. These stages have distinct technical meanings—and founders often blur them. By the end, you should be able to ask what was actually trained, how it was validated, what has truly been deployed, and what happens every time a customer uses the product.

---

Four stages, four different questions

The terms are easiest to retain if each answers a different question:

Stage	Core question	Main output
Training	How does the model acquire or change its behavior?	Updated model parameters, often called weights
Validation	Is this candidate model good enough to trust relative to a standard?	Evidence supporting, delaying, or rejecting release
Deployment	How is the approved model made available safely and reliably in the product?	A production-ready model service and its surrounding system
Inference	What does the already-deployed model produce for this particular new input?	A score, classification, generated response, embedding, or other output

The boundary between training and inference is particularly important:

• During training, the system changes its parameters in response to data.
• During inference, the parameters are normally fixed. The model applies what it learned to a new request.

The diagram below is intentionally high level, but its division is useful: training culminates in a model; inference is the operational use of that model.

A traditional software analogy is helpful up to a point. Deployment resembles shipping a new version of an application to production. But unlike ordinary application code, a model’s behavior is substantially shaped by data and learned numerical parameters. A new model release may therefore change because the data changed, the training procedure changed, or the model architecture changed—even where the surrounding product code did not.

---

Training: changing the model from examples

Training is the process through which an algorithm adjusts model parameters to perform better on examples. In a predictive system, those examples commonly pair inputs with known outcomes:

• historical borrower information and subsequent repayment outcomes;
• prior transactions and fraud labels;
• support tickets and their eventual resolution category.

For a language model, the broad pretraining task is commonly to predict missing or next pieces of text across an enormous corpus. In later lessons, we will trace that mechanism more closely. For now, the key point is that training produces a model artifact: a particular set of learned weights, associated configuration, and usually a record of the data and code used to produce it.

Training is not synonymous with “using data” or “giving the model context.” Three activities that founders may casually call training are materially different:

Activity	Does it alter the model’s weights?	Typical purpose
Pretraining	Yes	Build broad capabilities from very large datasets
Fine-tuning	Yes	Modify a base model’s behavior for a narrower domain or task
Prompting	No	Tell a fixed model what to do for this request
Retrieval-augmented generation	No, ordinarily	Supply relevant external information at request time
Logging user interactions	Not immediately	Create evidence or data that may later support training

If a legal-tech company sends documents and instructions to a third-party LLM API, retrieves relevant clauses from its database, and asks the model for a summary, it may have a strong product. But it has not necessarily trained a model. Its most important work may instead be workflow design, document retrieval, prompting, evaluation, permissions, and quality controls.

That distinction is not semantic pedantry. Training usually entails data rights, experimentation, compute expense, model-version management, and an evaluation discipline. A product that depends only on prompting faces a different set of technical risks and opportunities than one that has demonstrably improved a model using a proprietary dataset.

Training is an investment, not automatically an advantage

For seed-stage diligence, “we train on customer data” should invite several follow-ups:

1. What is the unit of training data? A document, customer interaction, completed workflow, accepted draft, correction, transaction, or another unit?
2. What makes it a useful learning signal? A raw interaction log is not necessarily a reliable label of quality.
3. Do they have the rights and permissions to use it? This affects both legal risk and whether the claimed data asset is real.
4. What behavior improves when the data is used? Accuracy on extraction, ranking quality, adherence to a format, acceptance rate, or something else?
5. Is retraining necessary yet? A team should be able to explain why a prompt, retrieval improvement, or deterministic rule is insufficient.

For many early AI applications, training a proprietary model is neither necessary nor economically rational. The right question is not “Do they train?” but what is the cheapest reliable mechanism for creating customer value, and what evidence shows that it works?

---

Validation: checking a candidate before trusting it

Training performance alone is weak evidence. A model can appear excellent simply because it has learned quirks, duplicates, or even direct copies of the examples it saw during training. The real question is how it performs on relevant cases it did not learn from.

This is why teams reserve separate data for validation and testing. The conventional development loop works as follows:

1. Train candidate models using the training set.
2. Measure candidates on a separate validation set.
3. Choose model settings and make design decisions using validation results.
4. Use a protected test set for a final, less-contaminated estimate of expected performance.

The validation set is not just a ceremonial checkpoint. It is where the team chooses among alternatives: model versions, features, prompts, retrieval methods, thresholds, or fine-tuning configurations. Repeatedly changing the product based on the same validation examples can cause the team to overfit its decisions to that set as well. This is why a genuinely held-out test set—or fresh production-style data—is valuable.

“Validation” has two related meanings

In conversations, validation can refer to either:

1. Dataset-level validation: evaluating alternatives on a held-out validation set while developing a model.
2. Release validation: deciding whether a candidate model version should replace the version currently in production.

These are related but not identical. The second is broader. A release decision might combine offline evaluation scores with safety checks, latency measurements, cost, regressions on critical customer cases, and sometimes a limited live experiment.

For a fraud model, validation might ask whether the new version preserves detection quality while not creating an unacceptable number of false positives. For a generative claims-processing assistant, it might ask whether the new version correctly extracts mandatory fields, follows the required schema, avoids unsupported statements, and remains within a response-time budget.

Notice that validation is not necessarily a binary statement that a model is “correct.” It is a decision under trade-offs. A model may be more accurate on average but still unacceptable if it fails on a high-severity customer segment, costs three times as much to run, or produces more unsafe outputs.

What counts as validation for generative products?

Generative outputs rarely have only one acceptable answer, so simple accuracy is often insufficient. A credible generative-AI validation process starts with a representative set of real product tasks and explicit scoring criteria. Depending on the product, these may include:

• factual support from supplied documents;
• extraction accuracy by field;
• completeness of a drafted workflow artifact;
• conformance to a required format;
• acceptance or edit rate by users;
• safety and policy adherence;
• latency and cost per completed task.

Human review is often necessary for nuanced criteria, but it should be structured. “Our team tried it and liked it” is not validation. A useful founder answer describes the evaluation set, the baseline being beaten, the rubric, and the failure categories that still block release.

---

Deployment: turning a model into a dependable product capability

A model can be trained and validated without being deployed. It may exist as a file in an experiment environment, or as an internal checkpoint whose results are promising but not yet operationally safe.

Deployment makes a chosen model version available for production use. For an API-based LLM product, deployment may mean configuring the application to call a selected provider model through a controlled service layer. For a company hosting its own model, it can mean packaging model weights and serving software, allocating compute, exposing an endpoint, and integrating it into the application.

In both cases, deployment involves substantially more than putting a model behind a button:

Deployment concern	Why it matters to the product
Versioning	The team must know exactly which model, prompt, retrieval configuration, and policy were used for a given result.
Input handling	Customer data must be authenticated, authorized, cleaned, and transformed consistently.
Reliability and capacity	A useful capability must work under normal demand, timeouts, provider failures, and spikes.
Guardrails and post-processing	Model output may need schema checks, safety filters, policy rules, or human escalation.
Observability	Teams need logs and metrics to identify failures, cost increases, quality regressions, and abuse.
Rollback	A problematic new version must be reversible without a long outage or manual scramble.

The final product output often differs from the raw model output. Recall the previous lesson’s fraud example: a model produces a risk score; explicit policy determines whether to block, approve, or send the transaction for review. Similarly, an LLM may draft a response, while the application validates citations, enforces permissions, and decides whether the draft can be sent automatically.

Google’s lifecycle description makes this operational distinction concrete.

Deployment does not mean the company trained the model

This is a common source of muddled claims in generative AI. A startup may deploy an excellent AI capability while relying entirely on a third-party foundation model. Its deployment work can still be technically significant: handling enterprise permissions, retrieval, workflow state, routing across models, output validation, monitoring, and integration with systems of record.

Conversely, a company may have trained a promising model but lack the deployment maturity to turn it into a dependable customer product.

When a founder says “the model is in production,” clarify the unit of analysis:

• Is it their own trained or fine-tuned model, a provider model, or a combination?
• Which model version is in use?
• Is it available to all users, a limited design-partner cohort, or only internal staff?
• What controls operate before and after the model call?
• What evidence exists from actual production tasks, rather than internal examples?

---

Inference: each live use of a fixed model

Inference is the runtime act of applying a deployed model to a new input. It is what happens when a user uploads a document, asks a copilot a question, receives a fraud decision, or runs a forecast.

For a classical predictive model, inference might be a single calculation: current account data enters, and the model returns a churn probability. For an LLM, inference is a sequence of computations that produces tokens one after another until the system stops. In both cases, the model is using its existing parameters rather than updating them.

Inference can happen in two broad operating modes:

Mode	What happens	Example
Online inference	A new request is sent to the model and a response is returned in real time.	A user asks a contract assistant to summarize an uploaded agreement.
Offline inference	Predictions are computed in advance, stored, and later looked up by the application.	A platform calculates churn scores nightly for all customers.

The choice is a product and economic decision. Real-time inference is necessary for an interactive assistant but exposes the user to latency and each request’s cost. Offline inference can make a user experience fast and predictable, but it only works when outputs can be computed in advance and remain useful long enough.

Inference is not learning, but it creates the next learning opportunity

A deployed model normally does not learn from a single customer request at the moment it answers. Its output may be logged, however, along with user edits, explicit ratings, eventual outcomes, failures, and operational context. This can feed monitoring and later training.

That creates a repeating lifecycle:

1. The product serves a model through inference.
2. The team observes quality, cost, latency, errors, and customer outcomes.
3. New data is processed into future training and evaluation datasets.
4. A candidate model or system configuration is trained or changed.
5. It is validated against explicit release criteria.
6. If approved, it is deployed as a new version.

The loop matters because the world changes. Fraud tactics evolve, product catalogues change, customer language shifts, and an AI application’s own users discover edge cases. This is usually called model staleness or drift when a model’s previous patterns become less useful. Not every model requires frequent retraining: the characteristics needed to identify a flower may be stable for years, while a transaction-risk model may need continual adaptation.

For generative-AI applications, some improvement cycles change no model weights at all. A company may deploy a better retrieval strategy, improve a prompt, add deterministic checks, modify routing to a different base model, or redesign the human-review point. It is still essential to validate such changes before release, because each can alter customer-facing behavior.

---

A worked product trace: AI-assisted investment-memo preparation

Consider a fictional tool that helps an investment team prepare an initial company brief from a data room, founder materials, and CRM history.

1. Training

A base LLM provider previously trained a general-purpose foundation model on broad text and code data. The startup may optionally fine-tune a smaller model to classify document types or extract a few highly standardized fields. That fine-tuning would be genuine training because it changes weights.

But suppose the startup does not fine-tune the LLM. It still builds a retrieval index over the current deal materials and writes instructions for producing a structured memo. Those are application design choices, not LLM training.

2. Validation

Before changing its memo workflow, the team assembles 100 representative historical deals. It evaluates whether the new system:

• identifies the correct company and financing facts;
• grounds claims in available source material;
• separates missing information from negative evidence;
• produces the required memo structure;
• reduces analyst preparation time without increasing serious factual errors.

A change that makes prose more polished but causes more unsupported market claims should fail validation, even if a generic language benchmark improved.

3. Deployment

The approved version is released with a particular base-model choice, retrieval configuration, prompt version, permissions layer, citation display, and rule that prevents automatic inclusion of unverified financial figures. The company logs source retrieval, model outputs, user edits, latency, and cost. It can roll back if the new version performs poorly.

4. Inference

When an investor opens a fresh deal, the system retrieves materials associated with that deal and calls the deployed LLM configuration. The generated brief is one inference result. If the investor corrects a revenue figure, that correction does not instantly retrain the LLM. It is evidence the company may later use to diagnose retrieval, prompt, source-data, or model shortcomings.

This trace shows why customer value does not live only in “the model.” A production AI system is a combination of model capabilities, current context, deterministic controls, interface design, and operational discipline.

---

A founder-conversation card

Use these prompts to keep the terms separate in a diligence conversation.

If a founder says…	Clarifying question
“We trained our AI on proprietary data.”	Which model’s weights changed, what data was used, and what measurable behavior improved?
“The model has been validated.”	Against which representative cases and criteria? What baseline did it beat, and what failures remain?
“It is deployed.”	Where is it live, for whom, with what controls, versioning, monitoring, and rollback?
“Our AI learns from every customer.”	Does it change immediately at inference, or are interactions logged for a later, reviewed training cycle?
“We use a model in production.”	Is it a proprietary model, a fine-tuned model, a third-party foundation-model API, or a hybrid system?

A good answer does not need to be elaborate. It should be precise about the model asset, the evidence, and the operational boundary. Vagueness often indicates that a team is still moving from a compelling demo toward a maintainable product.

---

Key takeaways

The lifecycle terms identify distinct activities:

• Training changes model parameters using data.
• Validation assesses a candidate model or system configuration against criteria that matter before release; held-out examples are essential.
• Deployment makes an approved version reliably available in a real product, with controls, monitoring, and rollback.
• Inference is each runtime use of the deployed model on a new input; it normally applies fixed parameters rather than learning immediately.

For investment diligence, distinguish training from prompting, retrieval, and logging; distinguish a validated experiment from a deployed customer capability; and distinguish deployment from the individual inference calls users experience.

Next, we will open the training stage itself and trace one neural-network update: input, prediction, loss, and the small parameter adjustment that enables learning.