Function Calling for LLMs: Interacting with External APIs

Hello! Welcome to the second lesson in our module on Agentic AI Systems.

In our last lesson, we designed a basic agent architecture centered on the perception-action loop. We established that an agent's ability to act on its environment depends on having a set of "tools." Today, we will bridge the gap between that theoretical design and practical implementation.

Your learning outcome for this lesson is to implement function calling (tool use) to allow an LLM to interact with external APIs. This is the core mechanism that transforms a passive language model into an active agent capable of gathering information, interacting with other software, and taking actions in the digital world.

1. Why Do LLMs Need Function Calling?

A standard Large Language Model is a powerful text generator, but it has two fundamental limitations:

1. Knowledge Cutoff: It only knows about the data it was trained on, which becomes outdated. It has no access to real-time information like today's weather, stock prices, or the latest news.
2. Inability to Act: It cannot perform actions outside of generating text. It can't send an email, query a database, or execute code.

For an LLM to become the "brain" of an agent, we need a reliable way for it to access live data and trigger external actions. Early attempts involved complex prompt engineering to coax the model into generating structured output (like JSON), but this was often brittle and unpredictable.

Function calling solves this by providing a robust, model-native way for an LLM to signal its intent to use a tool. Instead of generating free-form text, the model is fine-tuned to output a structured JSON object that explicitly names the function to call and the arguments to use.

2. The Function Calling Workflow

The process of function calling is a concrete implementation of the perception-action loop we discussed previously. It's a multi-step conversation between your application and the LLM.

The workflow unfolds as follows:

1. Define Tools & User Prompt: Your application sends the user's query to the LLM. Crucially, you also send a list of available "tools"—descriptions of functions the LLM can ask to call.
2. LLM Plans an Action: The LLM analyzes the user's query and the available tools. If it determines that one of the tools can help answer the query, it doesn't answer directly. Instead, it outputs a special message containing a JSON object with the name of the function it wants to call and the arguments it has extracted from the query.
3. Application Executes the Action: Your code receives this special message. You parse the JSON, identify the requested function (e.g., get_current_weather), and execute your actual Python function with the provided arguments (e.g., city="Tokyo"). The LLM does not run the code; your application does.
4. Observe & Summarize: You take the return value from your function (e.g., "15°C and sunny") and send it back to the LLM in a new API call. This message informs the model about the result of the tool it asked to use.
5. LLM Generates Final Response: With the information from the tool, the LLM now has the context it needs to generate a natural language response for the user (e.g., "The weather in Tokyo is currently 15°C and sunny.").

3. Implementation with the OpenAI API

Let's translate this workflow into Python code. We'll use the openai library to build an agent that can answer questions about currency exchange rates and search the internet.

3.1. Step 1: Defining the Tool Schemas

First, we need to describe our functions to the LLM. This is done using a specific JSON schema format. For each tool, you must provide:

• type: Always "function".
• function: An object containing:
  • name: The exact name of your Python function.
  • description: A clear, natural language explanation of what the function does. This is the most important part, as the LLM uses this description to decide when to use the tool.
  • parameters: A JSON schema object describing the function's arguments, including their type and a description for each.

3.2. Step 2-5: The Full Loop in Code

Now we'll implement the complete perception-action loop. This involves an initial API call to get the LLM's plan, executing the plan, and a second API call to get the final summary.

The process looks like this in Python:

1. First API Call:

   • Define your list of tools (the JSON schemas from the previous step).
   • Define your messages (e.g., a user query like "How much is a dollar worth in Japan?").
   • Call client.chat.completions.create with the model, messages, and the tools list.

2. Parse Response & Execute:

   • The response will not contain a text message. Instead, it will have a tool_calls attribute.
   • Iterate through the tool_calls. For each one, get the function name and arguments.
   • Call your local Python function with these arguments.

3. Second API Call:

   • Append the LLM's response (containing the tool_calls object) to your messages list.
   • For each tool you executed, append a new message with role="tool", the tool_call_id, and the content (the return value from your function).
   • Call client.chat.completions.create again with this updated messages list. This time, the response will be a natural language answer for the user.

{
    "type": "function",
    "function": {
        "name": "get_stock_price",
        "description": "Retrieves the current stock price for a given stock ticker symbol.",
        "parameters": {
            "type": "object",
            "properties": {
                "ticker": {
                    "type": "string",
                    "description": "The stock ticker symbol, e.g., 'AAPL' for Apple or 'GOOGL' for Google."
                }
            },
            "required": ["ticker"]
        }
    }
}

4. Robust and Scalable Tool Definition with Pydantic

Manually writing and maintaining JSON schemas is tedious and error-prone. Given your background in software engineering, you'll appreciate a more robust, programmatic approach. This is where Pydantic comes in.

Pydantic is a Python library for data validation and settings management using Python type annotations. You can define a Pydantic BaseModel to represent the arguments for your function, and Pydantic can automatically generate the corresponding JSON schema for you.

This approach has two major benefits:

1. Single Source of Truth: Your Python function signature and the schema passed to the LLM are derived from the same Pydantic model. If you update the model, the schema updates automatically.
2. Type Safety and Validation: When the LLM returns arguments, you can parse them directly into your Pydantic model, which automatically validates the data types and structure.

The video below demonstrates this in practice, showing how to define a very complex send_email_campaign tool using a Pydantic model, making the schema generation trivial.

5. Abstracting the Workflow with Frameworks

You may have noticed that the function calling format is specific to OpenAI. Other model providers like Anthropic (Claude) and Google (Gemini) also support function calling, but with slightly different API schemas. Writing provider-specific code for each can be cumbersome.

Frameworks like LangChain provide a standardized, model-agnostic interface for defining and using tools. You define your tool once, and LangChain handles the conversion to the appropriate format for whichever backend LLM you choose.

Conclusion

In this lesson, you have moved from a theoretical concept of "tools" to a full, practical implementation of function calling. You now have the foundational skill to build AI agents that can break out of the LLM's confines and interact with the digital world.

Key Takeaways:

• Function calling is the mechanism that allows an LLM to request the execution of external functions, enabling it to access live data and perform actions.
• The workflow is a two-step API call process: the first call gets the LLM's plan (which tools to call), and the second call gets the final answer after the tools have been executed.
• Tool schemas, especially the description fields, are crucial for the LLM to understand what each tool does and when to use it.
• Using Pydantic to define argument schemas is a best practice that ensures consistency and simplifies development.
• Frameworks like LangChain provide a higher-level, model-agnostic abstraction over the raw function calling APIs.

Preview of the Next Lesson:

We've given our agent "hands" by implementing tool use. However, its reasoning is still simple. In the next lesson, we will explore the ReAct (Reason + Act) prompting framework. This powerful technique enables an agent to "think out loud," combining chain-of-thought reasoning with actions to solve more complex, multi-step problems.