Stop Building AI Agents Like It's 2023

Listen. I've shipped 14 AI agent projects this year. Half of them shouldn't have worked. But they did because we stopped cargo-culting Medium tutorials and started thinking.

You're probably making the same mistakes we made in early 2024. Let me save you some pain.

Mistake #1: Using LangChain Because Everyone Else Does

I'll be blunt: stop using LangChain for new projects.

It made sense in 2023 when we had no alternatives. But LangChain has become the jQuery of AI, a bloated abstraction that hides what's actually happening.

What we use now:

For simple agents: Vercel AI SDK + raw API calls

import { generateText, streamText } from "ai";
import { anthropic } from "@ai-sdk/anthropic";

// That's it. No middleware hell.
const result = await generateText({
  model: anthropic("claude-opus-3-5"),
  messages: [
    { role: "system", content: "You're a helpful assistant" },
    { role: "user", content: userMessage }
  ],
  tools: {
    searchWeb: {
      description: "Search the web",
      parameters: z.object({ query: z.string() }),
      execute: async ({ query }) => webSearch(query),
    }
  }
});

Clean. Debuggable. Fast.

For complex multi-agent systems: LangGraph (from the LangChain team, but actually good)

import { StateGraph } from "@langchain/langgraph";

const graph = new StateGraph({
  channels: { messages: [], currentAgent: null }
});

graph.addNode("classify", classifyIntent);
graph.addNode("research", doResearch);
graph.addNode("respond", generateResponse);

graph.addConditionalEdges("classify",
  (state) => state.currentAgent
);

LangGraph gives you explicit control. You see the graph. You debug the graph. You don't fight abstractions.

Mistake #2: Treating Every Problem Like a Chat

Not everything needs conversational context.

Real example: Client wanted an AI agent to analyze financial documents. They showed me a prototype with a 50-message conversation history for context.

The problem: Each request sent 12KB of conversation history with every API call. Costs were insane.

The fix: Realized they didn't need conversation at all. Just stateless document analysis.

// Before: Bloated
const messages = [
  ...last50Messages,  // 12KB
  { role: "user", content: `Analyze this: ${document}` }  // 4KB
];

// After: Lean
const messages = [
  {
    role: "user",
    content: `Analyze this financial document:\n\n${document}\n\nProvide: revenue, expenses, profit margin.`
  }
];

Result: 75% cost reduction. Same output quality.

When you actually need state:

Use a proper state store. Redis for session data. Database for long-term memory.

import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

// Store conversation efficiently
await redis.set(
  `conv:${userId}`,
  JSON.stringify(messages.slice(-10)),  // Last 10 only
  { ex: 3600 }  // Expire after 1 hour
);

Mistake #3: No Error Handling (aka "The Demo Works" Syndrome)

Your demo works because you test happy paths. Production is chaos.

Things that WILL break your AI agent:

API rate limits - Claude has rate limits. OpenAI has rate limits. Your app will hit them.
Context length overflow - Users will paste entire codebases into your chat.
Malformed tool calls - The LLM will occasionally return invalid JSON for tool calls.
Timeouts - Some requests take 30+ seconds. Your infrastructure times out at 10.

How we handle this:

Exponential backoff with jitter:

async function callWithRetry<T>(
  fn: () => Promise<T>,
  maxRetries = 3
): Promise<T> {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      if (i === maxRetries - 1) throw error;

      if (error.status === 429) {  // Rate limit
        const delay = Math.min(1000 * Math.pow(2, i) + Math.random() * 1000, 10000);
        await sleep(delay);
        continue;
      }

      if (error.status >= 500) {  // Server error
        const delay = 1000 * Math.pow(2, i);
        await sleep(delay);
        continue;
      }

      throw error;  // Client errors don't retry
    }
  }
  throw new Error("Max retries exceeded");
}

Streaming with timeout protection:

export async function POST(req: Request) {
  const controller = new AbortController();
  const timeout = setTimeout(() => controller.abort(), 25000);  // 25s max

  try {
    const result = await streamText({
      model: anthropic("claude-opus-3-5"),
      messages: await req.json(),
      abortSignal: controller.signal,
    });

    return result.toDataStreamResponse();
  } catch (error) {
    if (error.name === "AbortError") {
      return new Response("Request timeout", { status: 504 });
    }
    throw error;
  } finally {
    clearTimeout(timeout);
  }
}

Mistake #4: Prompts That Try To Do Everything

Your system prompt is 2000 tokens of instructions. Half of them contradict each other.

Shorter prompts work better. Seriously.

Bad prompt:

You are a helpful, friendly, professional AI assistant that helps users with
their questions. You should be polite and courteous at all times. When answering
questions, make sure to be accurate and cite your sources when possible. If you
don't know something, say so. Don't make things up. Be concise but thorough.
Use examples when helpful. Format your responses clearly with markdown. Use
bullet points for lists. Use code blocks for code. Be sure to...

(continues for 1500 more tokens)

Good prompt:

You're an AI assistant. Be accurate, concise, and cite sources.

Let the model do its job. Claude 3.5 and GPT-4 don't need hand-holding.

When you need structured output:

Use actual structured output, not prompt engineering.

import { generateObject } from "ai";
import { z } from "zod";

const result = await generateObject({
  model: anthropic("claude-opus-3-5"),
  schema: z.object({
    sentiment: z.enum(["positive", "negative", "neutral"]),
    entities: z.array(z.string()),
    summary: z.string(),
  }),
  prompt: "Analyze this text: " + text,
});

// result.object is typed and validated
const { sentiment, entities, summary } = result.object;

No more parsing markdown tables. No more "please return JSON" prayers.

Mistake #5: Not Measuring What Matters

You're tracking token usage. Cool. What about:

Time to first token (perceived latency)
Success rate (% of requests that complete without errors)
Tool call accuracy (% of tool calls that are valid)
User retry rate (% of users who regenerate responses)

These metrics tell you if your agent actually works.

Simple instrumentation:

import { track } from "@vercel/analytics";

const startTime = Date.now();
let firstTokenTime: number | null = null;

const result = streamText({
  model: anthropic("claude-opus-3-5"),
  messages,
  onChunk: ({ chunk }) => {
    if (firstTokenTime === null) {
      firstTokenTime = Date.now();
      track("first_token_latency", {
        ms: firstTokenTime - startTime
      });
    }
  },
  onFinish: ({ usage, finishReason }) => {
    track("completion", {
      totalTime: Date.now() - startTime,
      tokens: usage.totalTokens,
      finishReason,
    });
  },
});

Look at your p95 first-token latency. If it's > 2 seconds, users are leaving.

Mistake #6: Treating AI Agents Like Microservices

AI is probabilistic. Your agent won't respond the same way twice. That's okay.

Stop trying to make AI deterministic with:

Temperature = 0 (reduces creativity, doesn't guarantee consistency)
Massive validation logic (the LLM can handle this)
10 retries until the output exactly matches your regex (just fix your regex)

Instead, embrace the variance. Test with multiple runs. Accept "good enough."

We run every agent against a test suite of 50 queries, 3 times each (150 total). If 90%+ pass, we ship.

What Actually Works in 2025

Here's our current stack for production AI agents:

Core:

Claude Opus 3.5 for reasoning + coding tasks
Gemini 3 Pro for speed-critical tasks (100ms first token)
GPT-4o when we need vision

Frameworks:

Vercel AI SDK for simple agents (85% of our use cases)
LangGraph for complex multi-agent workflows
Instructor (with Pydantic) for Python projects that need strict typing

Infrastructure:

Vercel Edge Functions for low latency
Upstash Redis for session state
Trigger.dev for long-running background agents
Helicone or Langfuse for observability

Data:

Pinecone for vector search (fast, reliable, boring in a good way)
Neon Postgres for structured data with pgvector when we need it

One More Thing

The best AI agents are boring.

They do one thing well. They fail gracefully. They don't try to be AGI.

If your agent has 15 tools and a 3000-token system prompt, you've overengineered it.

Start simple. Add complexity only when users ask for it.

Ship fast. Iterate based on real usage, not hypothetical edge cases.

Need help shipping AI agents that don't suck? We've built a few. Talk to us.