Managing Claude API Limits and Errors

Welcome back! In our last lesson, you dramatically improved your bot's user experience by implementing real-time streaming responses. While the bot is now highly responsive, our current error handling is a simple try...catch block that treats every problem identically. A production-quality application, however, must be able to distinguish between different failure modes and react accordingly.

Today, we will build a robust error handling system for your Claude-powered bot. You will learn to identify and handle specific API errors like rate limits, token limits, and other request failures. This will not only make your bot more reliable but also enable it to provide clear, helpful feedback to the user when things go wrong, guiding them toward a solution. We'll also explore proactive strategies to prevent one of the most common issues: exceeding the API's context token limit.

Understanding the Landscape of API Errors

When an API call fails, it's rarely a generic event. The failure could be due to network issues, invalid credentials, temporary server overload, or, very commonly with LLMs, a problem with the request content itself. A robust system needs to understand the difference.

The Anthropic SDK is designed to help with this. Instead of throwing a generic Error object, it throws specific, typed error classes that correspond to different HTTP status codes returned by the API.

Let's consult the official documentation to see how the SDK structures these errors.

This is a significant improvement over a generic catch (e). By using instanceof, we can identify the exact reason for the failure and tailor our response. For example, we can tell the user to wait a moment during a RateLimitError, or we can suggest they rephrase their prompt during a BadRequestError that might be caused by content filtering.

The general Claude API documentation provides more context on what these error codes mean.

With this knowledge, we can start building a more intelligent catch block.

A More Robust Error Handling Pattern

Before we dive into the implementation, let's consider a powerful pattern for handling errors in modern TypeScript. Your extensive experience with frontend development has likely exposed you to the evolution of asynchronous programming and the challenges of error handling. The standard try...catch block is powerful but has a notable weakness: the error variable e in catch(e) is of type unknown or any, forcing you to perform manual type guards and checks.

A more explicit approach, inspired by languages like Go and Rust, is to return errors as part of a function's result instead of throwing them. This makes the possibility of failure explicit in the function's signature. The popular YouTuber and developer Theo provides an excellent overview of this concept.

While we won't add a new library like neverthrow today, we will adopt its core principle. Our goal is to catch specific Anthropic.APIError types and then provide targeted, helpful messages to the user.

Implementing Graceful Error Responses

Let's update the try...catch block in our bot.on("message:text", ...) handler. We'll inspect the error object and respond differently based on its type.

Here’s the logic we'll implement:

1. Check if it's an Anthropic.APIError.
2. If it is, use a switch statement on the error's status code (or check the instanceof the specific error class).
3. Handle key cases:
   • RateLimitError (429): Inform the user that the service is experiencing high traffic and to try again shortly. The SDK automatically retries a few times, so if the user sees this message, the overload is persistent.
   • BadRequestError (400): This can happen for several reasons, including a prompt that violates content policies or, more commonly, a conversation history that exceeds the model's token limit. We'll give a specific message for the token limit case.
   • InternalServerError (>=500) or OverloadedError (529): These indicate a problem on Anthropic's end. We can give a general "service is unavailable" message.
4. For any other error, provide a generic failure message.

Proactive Strategy: Managing Token Limits

The most common and complex error to handle is exceeding the token limit. As a conversation grows, the entire history is sent with each new prompt. This causes the token count—and cost—to grow quadratically. Eventually, it will exceed the model's context window (e.g., 200k tokens for Sonnet 3.5), resulting in a BadRequestError.

The best way to handle this error is to prevent it. We need to give the user a way to manage the conversation's context.

The following video, though made for a different tool, provides an excellent conceptual overview of why context management is critical and offers several effective strategies.

Inspired by this, we will implement the simplest and most effective strategy: a /new command that clears the conversation history. This gives the user direct control over the context window, allowing them to start a new "thread" whenever a topic changes or a conversation becomes too long.

We will also implement a simple, automatic truncation strategy as a fallback. In our handler, we will ensure the message history we send to Claude never exceeds a certain number of turns. This acts as a safety net to prevent runaway conversations from immediately hitting the token limit.

Putting It All Together

Let's integrate these changes into our src/bot.ts file. We will add a new command handler for /new and significantly upgrade the catch block in our message handler.

Here is the updated code for src/bot.ts:

// src/bot.ts

import { Bot, Context, session, SessionFlavor } from "grammy";
import { claude } from "./services/claude";
import Anthropic from "@anthropic-ai/sdk";

// --- Type Definitions ---
interface Message {
  role: "user" | "assistant";
  content: string;
}

interface SessionData {
  messages: Message[];
}

type MyContext = Context & SessionFlavor<SessionData>;

// --- Bot Initialization ---
const bot = new Bot<MyContext>(process.env.TELEGRAM_BOT_TOKEN!);

bot.use(session({
  initial: (): SessionData => ({ messages: [] }),
}));

// --- Bot Handlers ---
const resetSession = (ctx: MyContext) => {
  ctx.session.messages = [];
  return ctx.reply("Hello! I'm an AI assistant. I've cleared our previous conversation. How can I help you?");
};

bot.command("start", resetSession);
bot.command("new", resetSession);

bot.on("message:text", async (ctx) => {
  try {
    const userPrompt = ctx.message.text;

    // Proactive history management: keep only the last 10 messages (5 turns)
    const recentMessages = ctx.session.messages.slice(-10);
    recentMessages.push({ role: "user", content: userPrompt });
    ctx.session.messages.push({ role: "user", content: userPrompt });

    const placeholder = await ctx.reply("...");
    await ctx.replyWithChatAction("typing");

    let responseText = "";
    let lastEdit = Date.now();
    const EDIT_INTERVAL = 800;

    const stream = claude.messages.stream({
      model: "claude-3-5-sonnet-20240620",
      max_tokens: 1024,
      messages: recentMessages, // Send only the truncated history
    });

    // ... (streaming logic from previous lesson remains the same)
    for await (const chunk of stream) {
      if (chunk.type === "content_block_delta" && chunk.delta.type === "text_delta") {
        responseText += chunk.delta.text;

        if (Date.now() - lastEdit > EDIT_INTERVAL && responseText.length > 0) {
          try {
            await ctx.api.editMessageText(ctx.chat.id, placeholder.message_id, responseText);
            lastEdit = Date.now();
          } catch (e) {
             // Ignore "message is not modified" errors from Telegram, which can happen.
             if (!(e instanceof Error && e.message.includes("not modified"))) {
                console.error("Error editing message:", e);
             }
          }
        }
      }
    }

    if (responseText.length > 0) {
        await ctx.api.editMessageText(ctx.chat.id, placeholder.message_id, responseText);
    } else {
        await ctx.api.editMessageText(ctx.chat.id, placeholder.message_id, "I apologize, but I couldn't generate a response.");
    }
    
    ctx.session.messages.push({ role: "assistant", content: responseText });

  } catch (err) {
    console.error("Error processing message:", err);
    let userMessage = "I'm sorry, but I encountered an error. Please try again.";

    if (err instanceof Anthropic.APIError) {
      switch (err.status) {
        case 400: // BadRequestError
          if (err.message.includes("max_tokens")) {
            userMessage = "This conversation has grown too long for me to remember. Please use the /new command to start a fresh conversation.";
          } else {
            userMessage = "I was unable to process your request. It might be due to a content policy. Please try rephrasing.";
          }
          break;
        case 429: // RateLimitError
          userMessage = "I'm currently experiencing high traffic. Please wait a moment before sending another message.";
          break;
        case 500: // InternalServerError
        case 529: // OverloadedError
          userMessage = "The service is temporarily unavailable. Please try again later.";
          break;
      }
    }
    await ctx.reply(userMessage);
  }
});

// --- Error Handling and Startup ---
bot.catch((err) => {
  const ctx = err.ctx;
  console.error(`Error while handling update ${ctx.update.update_id}:`);
  console.error(err.error);
});

async function startup() {
  await bot.start();
  console.log("Error-handling bot started...");
}

startup();

Key Changes:

1. /new Command: We've added a new command /new that, along with /start, calls a resetSession function to clear the ctx.session.messages array.
2. History Truncation: Before calling the API, we now create a recentMessages array using ctx.session.messages.slice(-10). This simple but effective measure prevents the context from growing indefinitely, making BadRequestError due to token limits much less likely.
3. Advanced catch Block: The catch block is now much more intelligent. It checks if the error is an Anthropic.APIError and uses a switch statement on the err.status code to provide specific, helpful feedback to the user.
4. Streaming Edit Error: A small try/catch was added around editMessageText to gracefully handle a common, benign error from Telegram when a message edit doesn't actually change the content.

Run your bot with bun start and test the new behaviors. Try having a long conversation and see how the bot behaves. Use the /new command to reset the context.

Conclusion

In this lesson, you have elevated your bot from a functional prototype to a robust application. By moving beyond generic error handling, your bot can now diagnose specific problems and provide intelligent feedback, significantly improving the user experience and its overall reliability.

Key Takeaways:

• The Anthropic SDK provides structured, typed error classes (e.g., RateLimitError, BadRequestError) that allow for precise error handling.
• By checking the error's instanceof or status code, you can provide users with specific, actionable feedback.
• Token limits are a critical issue for conversational AI. Proactive context management is the best solution.
• Simple strategies like a /new command to clear history and automatic truncation of the conversation log are highly effective at preventing token limit errors.

Your Claude-powered AI assistant is now feature-complete, context-aware, responsive, and robust. You have all the core skills needed to build powerful bots. The final step is to move your bot from your local machine to the cloud so it can run 24/7. In our next and final module, we will cover exactly that: deploying your bot for production use with webhooks.