Hello! Welcome to your fifth lesson on Agentic AI Systems.
In our last lesson, you built a powerful, dynamic memory system for your agent. This was a crucial step, as it allows an agent to retain context, user preferences, and the results of past actions across sessions. This persistent memory is the foundation upon which more complex, long-term behaviors are built.
Today, we move from remembering to strategizing. Your learning outcome is to apply task decomposition and planning for multi-step agent workflows. A simple ReAct loop is effective for short-term tasks, but for a complex goal like "research the pros and cons of the Mamba and Transformer architectures and write a summary report," the agent needs a high-level strategy. It must break the problem down and execute a sequence of steps.
1. From Monolithic Task to Decomposed Steps
The core idea is simple but powerful: complex problems are easier to solve when broken into smaller, manageable pieces. For an LLM agent, this "task decomposition" is not just a good strategy; it's a necessity to overcome challenges like finite context windows and to improve the quality of reasoning.
Lilian Weng's blog post, 'LLM Powered Autonomous Agents', is a foundational text in this space. We'll start by reading the section that frames the necessity of planning and task decomposition.
Read the sections 'Agent System Overview', 'Component One: Planning', and 'Task Decomposition'. Focus on how task decomposition is presented as a primary component of an agent's 'brain'. Pay attention to the different ways decomposition can be achieved, from simple prompting (like Chain of Thought) to more structured methods.
As you just read, an agent needs to plan ahead. This involves two key activities:
- Decomposition: Breaking a large task into a series of smaller sub-tasks.
- Planning: Orchestrating the execution of these sub-tasks, potentially including error handling and replanning.
This process can be visualized as a control loop, where a planner module sits at a higher level than an executor module.
2. Planning Patterns: From Simple to Sophisticated
Once we have a set of sub-tasks, we need a strategy to execute them. This is where planning patterns come in. We will explore an evolution of three patterns, each improving upon the last.
Given your background in software engineering, you can think of this progression as moving from a simple, linear script to a more dynamic system involving asynchronous execution and dependency graphs.
Our main resource for this section will be a practical guide from LangChain on building planning agents with their LangGraph library. While the code is specific to their framework, the concepts are universal and provide excellent blueprints for any agent system.
2.1. The "Plan-and-Execute" Pattern
This is the most direct approach. An LLM first generates a complete plan (a list of steps), and then an executor carries out those steps one by one.
Let's start with the foundational 'Plan and Execute' pattern. This video segment from LangChain explains the core idea and its components.
Watch from 02:46 to 04:29. Focus on the three main components introduced: the Planner, the Executors, and the optional Replanner. Understand how this separates the high-level thinking from the low-level tool use.
The Plan-and-Execute pattern formalizes the ReAct loop into a more structured workflow, often visualized like this:
A key limitation here is that the initial plan is static. If step 3 requires information generated in step 1, the planner doesn't know this in advance. This often necessitates a "replanning" step after each action, which can be inefficient as it requires another expensive LLM call.
2.2. REWOO: Adding Dynamic Data Flow
The REWOO (Reasoning without Observation) pattern improves upon Plan-and-Execute by introducing a critical concept: variable substitution. This allows the planner to create a more dynamic plan where steps can explicitly depend on the outputs of previous steps.
This is analogous to how you might structure a data pipeline, where the output of one function becomes the input for another.
Now, let's look at how REWOO makes plans more dynamic and efficient. This next segment explains the concept and its implementation.
Watch from 10:21 to 16:59. Pay close attention to: Variable Substitution (10:21 - 13:02): Understand how the initial plan can now include placeholders (e.g., #E1) that refer to the outputs of earlier steps. This is the core innovation. Prompt Format (13:02 - 14:40): Notice how the planner's prompt is structured to generate these variable assignments. It assigns the output of a tool call to a variable. Execution (14:40 - 16:59): See how the executor resolves these variables at runtime by looking up the results of previously completed tasks.
By defining dependencies at the planning stage, REWOO significantly reduces the need for frequent replanning. The agent can execute a whole chain of dependent tasks without needing to consult the master planner LLM at each step.
Test your understanding!
Imagine an agent is tasked with: "Find the current CEO of Microsoft and then find their year of birth." How would a REWOO-style plan differ from a simple Plan-and-Execute plan?
Show answer
-
Plan-and-Execute: The plan might be: 1.
Search for 'current CEO of Microsoft'. 2.Search for the birth year of the person found in step 1. The agent would execute step 1, get "Satya Nadella," then have to replan or use its short-term memory to formulate the query for step 2. -
REWOO: The plan would be more explicit and self-contained:
#E1 = search("current CEO of Microsoft")#E2 = search(f"{#E1}'s birth year")
The executor can run this entire plan without returning to the planner. It executes step 1, stores the result "Satya Nadella" in variable#E1, then substitutes it into the query for step 2 before execution.
2.3. LLM Compiler: Parallel Execution and Streaming
The final evolution we'll look at is inspired by compiler design. If some sub-tasks are independent, why execute them sequentially? The LLM Compiler approach treats the plan as a Directed Acyclic Graph (DAG), enabling parallel execution.
This is the most advanced pattern, drawing direct parallels to concepts in concurrent programming and compilers. It's designed for maximum speed and efficiency.
Watch from 19:53 to 28:48. This is a dense but fascinating section. Focus on the two key optimizations: Parallel Execution via DAG (19:53 - 21:40): The plan isn't a linear list but a graph where each task explicitly lists its dependencies. The system can then schedule and execute independent tasks in parallel. Streaming and Task Fetching (21:40 - 28:48): This is a clever optimization. Instead of waiting for the LLM to generate the entire plan, a 'task fetching unit' parses the LLM's streaming output. As soon as a complete task (with its dependencies) is parsed, it's sent to the scheduler for execution, even while the LLM is still generating the rest of the plan.
This approach represents the state-of-the-art in agentic planning, turning the agent's workflow into a highly optimized, dynamically scheduled execution graph.
3. A Broader View: Decomposition as Semantic Query Optimization
So far, we have focused on agent-centric planning loops. However, there's a powerful, complementary perspective that will resonate with your data and systems background: viewing task decomposition as a form of query optimization for semantic tasks.
When we ask an agent to "summarize court transcripts for signs of judicial bias," we are essentially running a complex query against unstructured data. Just as a database query planner rewrites a SQL query for efficiency, we can "rewrite" a semantic query to improve accuracy and reduce cost.
How to Process Documents at Scale with LLMs
This talk by Hamel Husain provides a brilliant systems-level perspective on this idea. He introduces the concept of 'rewrite directives' for systematically decomposing complex data processing tasks.
Watch the following segments to understand this powerful analogy: Systematic Task Decomposition (12:39 - 20:24): This is the core of the idea. Understand the two main rewrite strategies: Data Decomposition: Breaking a large document into smaller chunks and running the task on each (a classic MapReduce pattern). Task Complexity Decomposition: Breaking a complex prompt with multiple requirements into several simpler prompts, each handling one requirement. Rewriting for Cost (20:24 - 23:12): See how these rewrite rules can also be used to optimize for cost, such as by fusing simple operations together to reduce LLM calls or replacing an LLM call with a simple Python function.
This perspective provides a formal framework for thinking about decomposition. Instead of ad-hoc prompting, you can develop a library of "rewrite directives" to systematically improve the performance and cost-effectiveness of your agents when they perform complex information processing tasks.
Conclusion
In this lesson, you have moved beyond simple, reactive agents to understand the principles of strategic planning and execution. By breaking down complex problems and orchestrating the execution of sub-tasks, agents can tackle far more ambitious goals.
Key Takeaways:
- Task Decomposition is Essential: It's the primary strategy for agents to handle complex problems, improving reasoning and avoiding context limits.
- Planning Patterns Evolve: We saw a clear progression from simple Plan-and-Execute, to the more dynamic REWOO with variable substitution, and finally to the highly efficient LLM Compiler which uses DAGs for parallel execution.
- Planning is a Solved Problem in many domains: Your CS background is a huge asset here. Concepts like dependency graphs (DAGs), parallel execution, and query optimization are directly applicable to building sophisticated agents.
- Decomposition is a Form of Optimization: By thinking of agent tasks as "semantic queries," we can use systematic "rewrite directives" (like data and task decomposition) to improve both accuracy and cost-efficiency.
Preview of the Next Lesson:
A plan is only as good as its execution. What happens when a step fails or the environment changes unexpectedly? While we touched on "replanning," the next level of sophistication is self-correction. In our next lesson, "Implement agent reflection and self-correction mechanisms for improvement," you will learn how to build agents that can critique their own performance, learn from their mistakes, and refine their plans and actions for better outcomes.