Good to see you again. In the previous lesson, you saw that a large language model generates text from tokens and the context supplied at inference time. That mechanism explains a central business reality: an answer can be fluent, relevant-sounding, and still be wrong, incomplete, or unsuitable for the workflow around it.
This lesson focuses on identifying those limitations before presenting generative AI as a solution. By the end, you should be able to take a proposed use case—such as a customer-service assistant, sales-email generator, or internal knowledge chatbot—and state: what could fail, why it matters in this workflow, and whether the use case needs a narrower scope, stronger controls, a different technology, or no AI at all.
A limitation is practical only when connected to a consequence
“Generative AI has risks” is true but not useful in a business-development conversation. A credible assessment connects a technical limitation to a business outcome.
Consider three proposed uses for the same language model:
- Draft internal meeting summaries for a sales team.
- Answer customers’ questions about contract terms in a public chat.
- Decide whether a customer is eligible for a credit facility.
All three may involve generating text. Yet an incorrect meeting summary can usually be corrected by an employee; a wrong contract answer may create customer disputes; and an incorrect credit decision can unfairly harm an individual and expose the organization to serious legal and reputational consequences.
So do not ask only, “Can the model do this?” Ask:
If the model produces an inaccurate, biased, insecure, slow, or unpredictable output, who is affected, what happens next, and can the error be detected before harm occurs?
The infographic below is a useful high-level reminder that errors from uncontrolled generative AI do not remain technical issues. They can become operational, legal, financial, and reputational problems.
The five labels in the infographic overlap. For example, a chatbot that invents a refund policy may create an accuracy problem first, but then lead to a customer complaint, a costly manual correction, and reputational damage. The useful unit of analysis is therefore not a generic list of risks; it is the specific failure path in a proposed use case.
The NIST AI Risk Management Framework profile offers a broader vocabulary for this work. It distinguishes risks associated with the model, the application, human behavior, and the wider ecosystem—and emphasizes that likelihood and impact vary by context.
Artificial Intelligence Risk Management Framework
Read the relevant parts of NIST's framework to gain a disciplined vocabulary for generative-AI limitations. The goal is not to memorize all categories, but to recognize that a risk depends on the use case, the people affected, the inputs and outputs, and the potential consequence.
In Section 2, “Overview of Risks Unique to or Exacerbated by GAI” (PDF pp. 4–11), begin with the risk context. Notice the framework’s lifecycle, scope, source, and time-scale dimensions. Then read the numbered risk list on the following pages, using the risk catalogue to identify the categories most relevant to ordinary enterprise proposals. Finally, focus on Subsections 2.2, 2.6, 2.7, 2.8, and 2.9: confabulation, harmful bias, human-AI configuration, information integrity, and information security. Pay particular attention to why false outputs matter, performance disparities and bias, and the security discussion.
The first limitation: plausible language is not verified truth
NIST uses the term confabulation for confidently presented false or erroneous content. In everyday business language, this is often called a hallucination.
This is not merely a minor wording issue. As you learned previously, an LLM predicts likely next tokens based on patterns in its training and supplied context. It does not inherently verify each claim against a current, authoritative business source.
A model may therefore:
- invent a policy detail that was never provided;
- state an out-of-date price, product feature, or legal condition;
- create a nonexistent citation, case study, customer reference, or regulation;
- combine facts from separate accounts incorrectly;
- contradict an earlier part of its own response;
- omit an important exception while sounding highly confident.
What makes hallucination more or less serious?
The severity depends on the task.
| Proposed task | Typical accuracy consequence | Practical assessment |
|---|---|---|
| Draft an internal brainstorming outline | Usually low; a human can revise it | Often suitable for generative AI |
| Summarize a recorded call for an account manager | Moderate; omissions may affect follow-up | Suitable with review and access to the source record |
| Answer external questions about pricing or contract terms | High; a wrong answer can create disputes | Use only with reliable approved sources, narrow scope, and escalation |
| Recommend medical treatment, legal advice, or credit eligibility | Potentially severe and consequential | Generative AI should not make an unreviewed final decision |
A practical question for discovery is:
What source would a competent employee consult before giving this answer, and can the generated response be checked against that source before it is acted on?
If the answer is “there is no reliable source” or “the system must answer instantly without review,” the use case has a serious reliability limitation.
Accuracy also has a freshness problem
Even when a model produces generally correct background information, business facts change: prices, policies, product availability, regulations, account status, and delivery plans all evolve. A sales assistant cannot safely assume that a language model “knows” the latest commercial terms simply because the organization uses that model.
This does not mean generative AI is unusable. It means the scope must be honest. A system that drafts an answer from approved current material is fundamentally different from one that is expected to answer any question from general model knowledge.
Context is bounded, incomplete, and sometimes misleading
The previous lesson distinguished a model’s learned parameters from the context available for one request. In a business setting, this creates a second group of limitations.
A model cannot reliably use a CRM record, contract repository, product database, or customer email unless the application deliberately provides the relevant information and the model is permitted to access it. Even then, the supplied context may be:
- incomplete;
- outdated;
- contradictory;
- too long to include efficiently;
- poorly structured;
- inaccessible because of permissions;
- irrelevant to the user’s real question.
Suppose a customer-support assistant is asked, “Can I cancel without a fee?” The answer may depend on the customer’s contract, plan type, renewal date, geography, prior amendments, and a current policy exception. A generic cancellation-policy document may not be enough.
The limitation is not simply “the model has a context window.” The business limitation is:
The system may produce a confident generalized answer when the correct answer depends on account-specific information that is missing, inaccessible, or ambiguous.
For proposed use cases involving enterprise knowledge, identify:
- The authoritative source. Which system or document is the source of record?
- Source quality. Is it accurate, current, complete, and consistently maintained?
- Access boundary. Which users and systems should be allowed to see the information?
- Required specificity. Is a general answer acceptable, or does the answer need account-level precision?
- Fallback path. What happens when the source does not contain a clear answer?
A well-scoped assistant can say, “I could not find an approved answer; please route this to the contracts team.” That is often more valuable than a system that fills a gap with an invented answer.
Generative output is variable and not naturally deterministic
Traditional workflow automation is often valued because the same valid input produces the same traceable output. A rule such as “route invoices above the approval threshold to Finance” can be tested, audited, and changed precisely.
Generative AI behaves differently. Small changes in wording, context, model version, or generation settings can alter the response. The system may follow the intended structure most of the time, yet occasionally:
- use an inappropriate tone;
- ignore a formatting instruction;
- miss a business constraint;
- generate more or less detail than requested;
- choose a different interpretation of an ambiguous instruction.
This variability can be helpful in creative tasks such as drafting alternative campaign copy. It is much less helpful when the task requires a fixed calculation, a mandatory compliance statement, or a precisely formatted output for a downstream system.
A useful distinction is:
| Need in the workflow | Better fit |
|---|---|
| Produce a tailored draft, summary, explanation, or conversational response | Generative AI may be valuable |
| Apply a stable eligibility rule | Rules or conventional software are usually better |
| Predict a numeric outcome, such as likely demand or churn | A predictive model may be more appropriate |
| Calculate tax, commission, price, or inventory levels | Deterministic systems and validated formulas are usually better |
| Optimize schedules, routes, or allocation | Optimization methods may be a better fit |
IBM Technology’s short video makes the commercial point directly: popularity is not evidence that generative AI is the right tool.
Select the right AI use case for your business
Watch IBM Technology’s “Select the right AI use case for your business” for a concise comparison between generative AI and alternative approaches. It reinforces the discipline of starting from the business problem rather than forcing generative AI into every workflow.
Watch the opening warning on why generative AI does not fit every business scenario. Then watch the examples, especially the distinction between budget forecasting, scheduling, and generative tasks. Finish with the selection factors and note the four considerations: requirements, organizational capability, integration, and skills.
For business development, this is an important credibility test. Do not position generative AI as a replacement for a proven deterministic process merely because an LLM could imitate the final text output.
For example, an AI may draft an email explaining why a payment was declined. But the system that decides whether the payment is declined should ordinarily remain a rules-based or otherwise validated decisioning process. The generative component communicates or summarizes; it should not quietly become the source of financial truth.
Bias, inappropriate content, and uneven performance
Generative AI learns patterns from large amounts of human-created material. That material can include stereotypes, exclusionary language, uneven representation, and historical bias. As a result, generated content can be unfair, culturally inappropriate, or less reliable for particular groups, languages, dialects, and contexts.
In enterprise use, this can appear as:
- recruitment text that subtly discourages particular groups;
- sales or support language that stereotypes a customer;
- generated images that repeatedly represent leaders or professionals in narrow ways;
- poorer response quality in languages common among important customers or employees;
- inconsistent sentiment or quality analysis across accents, dialects, or regional communication styles.
Do not treat bias only as an ethical abstraction. Connect it to a workflow.
For a marketing-copy assistant, the question is whether the output could exclude or offend an audience. For a multilingual contact center, the question is whether service quality differs across customer language groups. For a talent process, the question is whether generated recommendations or evaluations could influence a consequential employment decision.
A practical limitation statement might be:
“The proposed assistant may generate language that is inconsistent with the organization’s inclusion standards or perform unevenly across the customer languages in scope. We should not claim equivalent quality across languages until the proposed workflow has been evaluated with representative examples.”
That phrasing is specific, commercially relevant, and avoids pretending that an abstract promise of “bias-free AI” is credible.
Privacy, security, intellectual property, and compliance are design constraints
Generative AI proposals commonly involve sensitive internal information: customer data, sales notes, financial records, source code, strategy documents, employee records, contracts, or product roadmaps. The model itself is not the only concern. Risk can arise from what users enter, what enterprise sources the application connects to, what appears in the output, and what the system is permitted to do next.
Privacy and confidentiality
Potential issues include:
- employees pasting sensitive data into an unapproved AI tool;
- a chatbot exposing information to a user who should not have access to it;
- an assistant combining information from different accounts or customers;
- retaining prompts or outputs longer than policy permits;
- using personal data for a purpose the organization has not authorized.
A proposal to “connect every internal document to the chatbot” should immediately prompt questions about classification, permissions, and approved use. More data is not automatically better data.
Security
When an AI application reads documents, connects to systems, or takes actions, it can create new paths for misuse. One example is prompt injection, in which malicious instructions are placed in a user prompt or in a document the system later reads. The aim can be to make the system disregard its intended instructions, reveal information, or use connected tools inappropriately.
At this stage, the essential business-development conclusion is simple:
A model that can read sensitive content or initiate actions must be assessed as part of the wider application and integration environment—not as an isolated chat interface.
Intellectual property and compliance
Generated output can create concerns about copyrighted, licensed, trademarked, or confidential material. Legal and regulatory requirements also differ by industry, geography, data type, and use case. A customer-facing assistant in financial services or healthcare deserves a more demanding assessment than an internal ideation tool.
You do not need to give legal advice in a sales conversation. You do need to recognize when a claim such as “we will upload all our customer contracts” or “the AI will write regulated customer communications automatically” requires specialist review.
IBM’s discussion of trustworthy AI illustrates why organizations connect hallucinations, data exposure, harmful content, and governance rather than treating them as isolated concerns.
Trust, transparency, and governance in the age of generative AI
Watch selected chapters of IBM Technology’s “Trust, transparency, and governance in the age of generative AI.” The examples show how apparently capable generated output can still create accuracy, confidentiality, fairness, and trust concerns.
Watch the hallucination example, where a chatbot generates a mixture of accurate and fabricated biographical details. Then watch the new risks for the connection between generative AI, sensitive-data exposure, and the difficulty of reviewing vast-scale training material. If your proposed use case affects customers or employees, also watch bias and governance, focusing on uneven effects across groups and the need to assess systems rather than simply trust them.
Cost, speed, and operational burden can undermine the business case
A generative-AI demonstration may look effortless, but production use has operating constraints.
From the prior lesson, recall that more input tokens require more processing and that outputs are generated sequentially. In practical terms, long source documents, lengthy conversations, large user populations, and high volumes of generated text can increase response time and usage costs.
The AWS contact-center guidance summarizes the operational issue well: generative AI can be slower and more expensive than simpler automation or purpose-built AI models. It also argues for working backwards from the customer problem rather than starting with a preferred technology.
Read AWS Contact Center’s practical comparison of manual work, rules, conventional AI, and generative AI. Although the examples are from contact centers, the selection logic applies to sales, operations, finance, and customer-service proposals more broadly.
In the section “Areas of challenges,” read the challenge summary. Focus on the linked concerns of accuracy, cost, speed, bias, and governance. Then read “Working backwards: How to determine if it’s a good use case for generative AI.” Compare the alternatives carefully: the rules alternative explains why simple repeatable flows may not need generative AI, while the traditional AI comparison explains why a smaller purpose-built model can be preferable. Finish with the generative-AI examples and the warning that outputs need human validation.
When reviewing a proposed use case, identify operational limitations through questions such as:
- How many requests will occur in a normal month and at peak times?
- How much source text must be processed for each request?
- What response time will users accept?
- Is a human review step required, and does that reduce the expected time saving?
- Will model usage, integration, monitoring, and support costs exceed the value created?
- Could a simpler workflow, rule, classifier, or search interface achieve most of the outcome?
A model that generates a polished monthly report in 20 seconds may be useful. A customer-service assistant that takes 20 seconds to answer a simple account question may create a poor customer experience and drive users back to human agents.
A repeatable limitation assessment for any proposed use case
Use the following screen whenever an AI opportunity is proposed. It helps turn broad concerns into evidence-based qualification.
| Assessment area | What to identify | Example warning signal |
|---|---|---|
| Task fit | Whether the core need is generation, prediction, calculation, classification, or workflow execution | The desired output is a fixed policy decision or numeric calculation |
| Accuracy requirement | What must be factually correct and how errors will be detected | A wrong answer is sent directly to a customer or used in a regulated decision |
| Source grounding | The authoritative information required for a correct answer | Product policies are scattered, outdated, or contradictory |
| Variability and control | Whether wording, format, and behavior can safely vary | Every answer must contain exact mandatory wording |
| Fairness and appropriateness | Which groups, languages, and audiences could be affected differently | The tool communicates with a multilingual or diverse customer base |
| Privacy and security | What data enters the system, who may see it, and what integrations exist | Users may enter customer data into an unapproved public tool |
| Operational viability | Volume, latency, cost, support burden, and review effort | High-volume simple work needs instant low-cost responses |
| Human role and accountability | Who reviews, approves, corrects, and owns the outcome | Staff are expected to accept output without checking it |
The result should not be “AI is risky.” It should be a concise use-case statement such as:
“Generative AI is potentially suitable for creating first-draft customer follow-up emails from approved account notes. Its main limitations are inaccurate account details, inappropriate commitments, and exposure of confidential CRM information. The initial scope should remain internal, use approved account information only, and require the account manager to review before sending.”
That is much stronger than promising an “autonomous AI sales agent.”
Worked example: a contract-support chatbot
Imagine a company proposes:
“Build a chatbot that answers customers’ questions about their contracts.”
At first glance, this sounds like a good generative-AI use case because customers ask questions in natural language and contracts are written in natural language. But the limitation assessment reveals important differences between possible scopes.
| Question | Finding | Implication |
|---|---|---|
| Does the answer require current, customer-specific information? | Often yes; contracts can vary by customer and amendment | General answers may be misleading |
| Are the consequences of a wrong answer material? | Yes; pricing, cancellation, liability, and delivery terms may be involved | Unreviewed answers create commercial and legal risk |
| Is the contract repository clean and permissioned? | Often uncertain | The system may retrieve the wrong version or expose another customer’s information |
| Can the model reliably interpret legal ambiguity? | No guarantee | The tool should not be positioned as legal advice |
| Can answers be checked before the customer acts? | Possibly, if the assistant serves internal teams first | Start with employee assistance rather than autonomous external advice |
| Is generative AI the only component required? | No; document access, permissions, search, and escalation are also necessary | The value proposition is a workflow solution, not simply a model |
A better initial proposal might be:
“Provide an internal assistant for account managers that locates relevant approved contract passages and drafts a plain-language explanation. It will clearly distinguish quoted terms from generated explanation and route ambiguous or high-risk questions to legal or contracts specialists.”
Notice what changed. The business objective—faster answers to contract questions—remains. But the scope acknowledges the model’s limitations rather than hiding them.
This is the professional standard in enterprise AI business development: articulate the value, identify the failure modes, and shape a use case that is proportionate to the risk.
Key takeaways
Generative AI is valuable when creating, summarizing, explaining, or transforming unstructured content is central to the business task. But its outputs are not inherently verified, complete, current, deterministic, unbiased, secure, or cost-effective at scale.
The most important practical limitations to test are:
- confabulation and factual inaccuracy;
- missing, stale, or conflicting business context;
- variable behavior and limited output control;
- bias, inappropriate content, and uneven performance;
- privacy, security, intellectual-property, and compliance exposure;
- human over-reliance and reputational consequences;
- latency, cost, integration, and operational burden;
- poor task fit when rules, conventional AI, optimization, or process redesign would work better.
A limitation becomes commercially useful when you connect it to the workflow: the affected user, the decision or action taken, the severity of error, and the available method for detecting or containing it.
Next, you will map common AI capabilities to business functions such as sales, operations, finance, and customer service. That will help you identify promising opportunities while keeping the limitations assessed here in view.
Can't find a good explanation? Sign up and we'll make it for you
Sign up