Evaluating Tasks for Generative AI Suitability

Hello. In the previous lesson, you separated what an LLM learned during training from what it receives as current context. That distinction is the foundation for sound task selection: generative AI can use an uploaded policy, a prompt, or retrieved records to produce useful work, but it does not thereby become a reliable authority on the policy.

This lesson develops a practical professional judgment: deciding when generative AI should draft or assist, when it can operate within tightly controlled limits, and when the task should remain primarily human work or use a conventional tool instead. By the end, you should be able to justify one of three decisions for a task: use generative AI with defined oversight, use it only as a limited assistant, or do not delegate the task to it.

---

Suitability is a property of the whole workflow

The question is not simply, “Can an AI produce something that looks like an answer?” For almost any language-based task, it probably can. The more useful question is:

Can we use this output safely and productively, given the task’s data, consequences, and available review?

Generative AI is particularly capable at working with language and other unstructured material. It can turn rough notes into a draft, summarize a document, classify feedback themes, explain a concept in different styles, extract candidate information from text, or propose variations on an idea. These are valuable capabilities because they reduce the effort of producing a first usable version.

But the same LLM that can write a convincing explanation can also:

• state a false claim confidently;
• omit an important qualification from its source material;
• misunderstand an ambiguous instruction;
• produce an answer that is polished but poorly suited to the organization’s actual context;
• reflect problematic assumptions found in its inputs or patterns;
• mishandle sensitive information if the workflow permits inappropriate access or sharing.

An LLM predicts plausible continuations based on its weights and supplied context. It does not carry legal responsibility, understand organizational priorities in the way a responsible person does, or independently know whether a seemingly plausible response has unacceptable consequences.

So, avoid two unhelpful extremes:

• “AI can do everything, so delegate it.”
• “AI can make mistakes, so it is never useful.”

A better position is to match the AI’s role to the risk and to build a workflow in which people retain responsibility for the decisions that matter.

Three practical roles for generative AI

Decision	Appropriate conditions	Example
Use as a controlled assistant	The task involves language, a draft or transformation is useful, and a person can check the result.	Drafting a customer reply from approved support notes.
Use with limited automation	Inputs, outputs, success criteria, and error handling are well-defined; the stakes are low or reversible; testing and monitoring are in place.	Categorizing routine, non-sensitive feedback into pre-defined themes for later review.
Do not delegate the task	The task requires high-stakes judgment, reliable truth without available evidence, confidential data without an approved process, or accountability for a consequential decision.	Automatically approving a loan, diagnosing a patient, or deciding who should be hired.

The third category does not always mean “never use AI anywhere near this work.” A clinician might use an approved system to help organize notes; an HR team might use a carefully reviewed tool to standardize job-description language. The boundary is that the model should not become the unaccountable decision-maker.

---

Start with capability fit

A good early use case uses the model for what it is actually good at, rather than treating it as a universal substitute for software, expertise, or judgment.

Anthropic’s Building trusted AI in the enterprise offers a useful checklist for selecting early AI projects: they should fit LLM strengths, have measurable success criteria, use available and permitted data, and avoid extreme security or operational risk.

Work that often fits well

Generative AI is often a strong first-pass worker when the input and output are language-based and the result can be checked against a clear standard. Typical examples include:

• summarizing a provided meeting transcript into action items;
• transforming a technical document into a beginner-friendly explanation;
• drafting a social-media post from approved campaign notes;
• turning messy customer comments into recurring themes;
• generating alternative wording, titles, email subject lines, or lesson examples;
• producing a first draft of code, documentation, or a test plan for an experienced person to inspect;
• extracting fields from a standard document format, with a human or deterministic validation step.

Notice what these have in common. They are usually bounded: there is a known input, an expected kind of output, and a person can explain what “good” looks like. The model is accelerating expression, organization, or transformation, not being asked to define the organization’s goals or accept the consequences of an error.

Work that needs another approach

Some tasks look like language tasks but are poor candidates for direct LLM delegation.

A conventional calculator, spreadsheet formula, database query, or rule-based system is usually better when the requirement is exact, repeatable computation. For example, use payroll software to calculate pay; use generative AI, if at all, to explain a payroll report in plain language after the numbers have been verified.

Similarly, a model cannot make an unsourced claim reliable merely by wording it fluently. If someone asks for the current legal requirement in a jurisdiction, current medical guidance, or the latest company policy, the workflow needs authoritative and current evidence. Giving the model a trusted source in its context can improve its answer, but it does not eliminate the need for verification.

Finally, tasks involving purpose, trade-offs, and values require more than a plausible response. A model can propose three product strategies; leaders still decide which strategy fits the organization’s priorities, budget, obligations, and appetite for risk.

---

The central test: consequences and verifiability

Two questions provide an unusually strong first judgment:

1. What happens if the output is wrong?
2. Can the relevant person detect that it is wrong before it causes harm?

A low-stakes output that is easy to verify is a good place to experiment. A mistaken draft title for an internal workshop is easy to spot and cheap to replace. A mistaken medication instruction, contract interpretation, credit decision, or public statement about a safety incident may be difficult to detect and expensive or harmful to correct.

This is why verifiability often matters as much as the model’s apparent capability. If it takes five seconds to check the output against a source, AI may save time. If checking requires a specialist investigation more demanding than doing the task directly, AI may add risk and cost rather than reduce it.

Anthropic’s research on how its own teams use Claude captures this practical delegation logic: people tend to delegate self-contained, repetitive tasks where validation is manageable, and they avoid spending more effort supplying context and correcting the AI than doing the task themselves.

A risk screen for individual tasks

Use the following dimensions together. A single red flag does not always prohibit AI use, but several red flags mean that you should narrow the AI’s role sharply or avoid using it.

Dimension	Lower-risk signal	Higher-risk signal
Task clarity	Clear input, output, constraints, and completion standard	Vague objective or substantial hidden organizational context
Evidence	Trusted source material is available and can be cited or checked	The output depends on unsupplied, current, disputed, or unknown facts
Impact of error	Easily corrected inconvenience	Financial, legal, medical, safety, employment, or reputational harm
Reversibility	Draft can be edited before release	Output triggers an irreversible action or reaches the public automatically
Verifiability	Reviewer can check quickly against a source or rubric	Only a specialist can detect subtle but serious errors
Data sensitivity	Non-sensitive or approved, minimized data	Personal, confidential, regulated, or proprietary data without an approved process
Autonomy	AI recommends, drafts, or organizes	AI decides, sends, purchases, deletes, or otherwise acts without approval
Efficiency	Prompting and checking are meaningfully faster than manual work	Context gathering and correction cost more than completing the task directly

The model’s output quality is only one row in this table. A highly capable model can still be the wrong tool if it lacks necessary context, handles data it should not receive, or is allowed to act beyond the safeguards of the process.

Human oversight must be real, not ceremonial

“Human in the loop” can become an empty phrase if the reviewer is expected to approve a result they cannot realistically assess. Effective oversight gives a person both authority and ability to intervene.

Depending on the task, oversight may include:

1. Before generation: defining the purpose, audience, constraints, allowed sources, and prohibited data.
2. During design: testing typical and difficult cases, setting access permissions, and deciding what the system may or may not do.
3. Before use: checking important factual claims, calculations, citations, tone, privacy, bias, and policy compliance.
4. After deployment: monitoring errors, collecting incidents, improving prompts or source material, and withdrawing the workflow if it proves unreliable.

For a low-risk, routine draft, a quick editor review may be enough. For a high-impact customer communication, a domain owner may approve each message. For a narrowly defined low-risk automated classification, it may be reasonable to review sampled outputs and all exceptions after careful testing. The necessary level of review grows with the severity and difficulty of detecting errors.

A key professional habit is recognizing the limits of your own ability to supervise. If you cannot judge whether an AI-generated legal, medical, security, or financial claim is correct, do not treat its confidence as evidence. Ask it to help formulate questions, identify relevant sources, or summarize expert-reviewed material; then consult an appropriate authoritative source or qualified reviewer.

---

Map, measure, manage, and govern the risk

The decision to use AI is not a one-time checkbox. The task, model, source material, organizational policies, and consequences can all change. The NIST AI Risk Management Framework provides a useful cycle for thinking about this work.

Think of the framework as four connected responsibilities:

• Map: Define the real task. Who uses the output? What information is involved? What could go wrong? What context does the model lack?
• Measure: Test the system against quality criteria and likely failure cases. Measure factual accuracy, format compliance, time saved, error rates, and unequal performance across cases where relevant.
• Manage: Choose safeguards based on what the testing reveals. Safeguards might include better source grounding, narrower scope, required review, redaction, access restrictions, or deciding not to deploy.
• Govern: Establish accountability. Someone owns the workflow, documents approved uses, handles incidents, and ensures that risk management remains part of routine practice.

The central placement of Govern matters. Governance is not merely a final approval step. It determines who may use the system, what data may be entered, who reviews outputs, and what happens when the system fails.

The following short video frames the same principle in human terms: models can produce useful material at high speed, but people still need to define the work, provide context, assess outputs, and make judgments about security and compliance.

---

Make and defend a decision

A professional AI recommendation should be more precise than “AI would be useful here.” Record the decision in a compact task assessment:

Assessment item	What to write
Task and intended user	A specific description, not a vague label such as “help with HR.”
AI role	Draft, summarize, extract, classify, explain, recommend, or act.
Required evidence and context	Approved documents, database records, style guide, policy, or source limitations.
Most important failure	The concrete error that would matter most.
Controls and reviewer	Who checks what, when they check it, and what they use as a standard.
Decision	Use with oversight, use only in a limited role, or do not delegate.

Consider three examples.

Example 1: Turn an approved policy into an employee FAQ

A team uploads the current, approved leave policy and asks the AI to draft a twelve-question FAQ for new employees.

This is a promising AI-assisted task. It is a language transformation grounded in a supplied document, and the output is a draft rather than a binding policy decision. The controls should require that the output cite or point to the relevant policy sections, avoid inventing exceptions, and receive review from the policy owner before publication. If the policy includes employee data, that data should be excluded.

Decision: Use generative AI with source-grounding and policy-owner review.

Example 2: Decide which job applicants should be rejected

A manager proposes uploading résumés and having an AI automatically select applicants to reject.

This is unsuitable for autonomous delegation. The decision has significant consequences for applicants and the organization; it can be affected by biased assumptions, incomplete evidence, and legal or policy obligations. It is also difficult to demonstrate that an opaque selection process treated candidates fairly.

There may be carefully designed, approved uses of AI in recruitment support, such as formatting job descriptions or organizing recruiter notes, but those are different from allowing a model to make an employment decision.

Decision: Do not delegate the decision. Restrict any supporting use to approved, transparent, non-decisional tasks with appropriate human accountability.

Example 3: Create five alternative subject lines for a newsletter

A communications coordinator provides the newsletter draft, target audience, and brand style, then asks for five subject-line options.

This is low stakes, easy to verify, reversible, and well-bounded. The coordinator can select or revise a result before sending anything. Still, the final message should remain subject to normal brand and factual review, especially if it contains claims about a product or event.

Decision: Use generative AI as a controlled assistant, with ordinary editorial review.

These examples demonstrate an important distinction: AI may contribute to a workflow even when it should not control the outcome. Its appropriate role can range from drafting language to organizing evidence, while humans retain ownership of the purpose, judgment, and consequences.

---

Key takeaways

A task is suitable for generative AI when its capabilities fit the work—especially drafting, summarizing, classifying, explaining, or transforming unstructured material—and when the resulting workflow has appropriate evidence, safeguards, and review.

Judge suitability across the whole workflow, not from the apparent intelligence of one response. Consider task clarity, current and permitted data, stakes, reversibility, verifiability, privacy, autonomy, and whether prompting plus checking truly saves effort.

The safest professional framing is usually: AI assists; people remain accountable. For low-risk, easily checked work, oversight can be light. As impact, uncertainty, sensitivity, or autonomy rise, the AI’s role should narrow and human review, testing, and governance should become stronger.

Next, you will choose among a Claude chat, a Project, and an Artifact for beginner use cases—applying this task judgment to select the right kind of workspace rather than treating every request as an ordinary chat.