How to Write a System Prompt for an AI Agent
A chatbot's system prompt sets a personality. An agent's is control logic the model rereads on every turn of the loop. Stop writing a persona and write a policy.
5 min
How to Ship an AI Agent Change Without Breaking It: Eval Gates, Shadow Replay, and Why Canaries Lie
You can't A/B test an agent the way you A/B test a button. The unit of variance is a trajectory, not a click — so the gate has to be offline, and "shadow mode" means something different than it does for a model.
4 min
How to Read an Agent-Memory Benchmark: The LoCoMo and LongMemEval Number Wars
Mem0 says 92.5% on LoCoMo. Mastra says 95% on LongMemEval. Zep corrected its own 84% to 58%. They can't all be right — and the baseline that beats them all is the one no vendor charts.
5 min
How to Evaluate an Embedding Model on Your Own Data
The MTEB leaderboard is a prior, not an oracle. The model that wins your RAG system is the one you measure on a few hundred of your own labeled queries — here is how to build that eval.
4 min
How to Evaluate a Deep Research Agent: Report Quality vs. Citation Accuracy
A deep research agent hands you a long, confident, well-structured report. Grading it means measuring two different things at once — how good it reads, and whether a single sentence is actually supported.
5 min
GLM-5.2 Matched the Closed Models on Agentic Coding — for a Sixth of the Cost
An open-weight model is now within a point of Claude Opus on long-horizon coding benchmarks. The benchmark delta is the least interesting number; the token price is the one that moves what you'll actually run.
4 min
Your Agent Is Now an MCP Server: What Exposing an Agent as a Tool Quietly Throws Away
Deploy a LangGraph agent and it auto-publishes a /mcp endpoint, so any client can call it as a tool. Convenient — and lossy. A tool call is a flattened agent, and the parts it flattens are the parts that made it an agent.
4 min
Eval-Driven Development: How to Ship an AI Agent Without Guessing
Write the eval before the prompt. The test suite you build first is the only thing that lets you change models next month without praying — and in 2026, you will change models.
4 min
The EU AI Act Deadline Didn't Really Move: What Still Hits AI Agents on August 2
The Digital Omnibus pushed the high-risk rules to 2027 — and most builders read that as a reprieve. But the deadline that actually catches a typical agent never moved at all.
5 min
Claude Agent SDK vs OpenAI Agents SDK: A Harness vs an Orchestration Library
Both vendors shipped an official agent SDK, so the choice looks like a feature bake-off. It isn't. They sit at different layers and bet on different hard parts — and their defaults decide which one your problem is one line of code away from.
5 min
Bedrock AgentCore vs Vertex Agent Engine vs Foundry Hosted Agents: The Managed Agent Runtime, Compared
All three hyperscalers now sell a managed home for your agent. Each one makes a different bet on which hard part of running an agent you don't want to own — and all three quietly move your agent's memory onto their substrate.
4 min
Autoscaling LLM Inference on Kubernetes: Scale on the Queue, Not the GPU
The metric you'd reach for first — CPU, then GPU utilization — is the one that lies. A 70B pod can read 5% CPU and a calm GPU dial while its request queue backs up for miles. Scale on queue depth instead.
4 min
Any-LLM vs LiteLLM: You're Comparing a Library to a Building
Mozilla's any-llm and LiteLLM get pitted against each other constantly, but they answer different questions — the only one that matters is whether you actually need a proxy.
5 min
Agent Sprawl: Why AI Agent Governance Now Starts With a Registry
Microsoft, Okta, and AWS all shipped the same first move against unmanaged agents — an inventory. It's the shadow-IT playbook again, except this time the thing you can't see replicates itself.
5 min
The AI AGENT Act, Explained: Warner's Bill Treats Blocking Your Agent as the Harm
A new Senate discussion draft reads like a privacy bill, but its teeth are an interoperability mandate — the first U.S. attempt to give users a right to bring an agent onto platforms that would rather keep it out.
4 min
A2A at One Year: Is Agent-to-Agent Interoperability Actually Happening?
The Agent2Agent protocol now claims 150-plus organizations and a slot in every major cloud. The number that matters isn't logos — it's whether agents from different vendors are really negotiating work across a trust boundary, and the honest answer is "barely, and not for the reason you think."
4 min
What Should an AI Agent's Tools Return? Designing Tool Results for the Context Window
Everyone tunes a tool's inputs — name, schema, description. The likelier production failure is the output: the right tool returns a payload that floods the model's context window.
4 min
When Should an AI Agent Compact Its Own Context? The Case Against Fixed Thresholds
Most agents summarize their context when a token counter trips. A 2026 result argues the counter is the wrong trigger — and that letting the model decide is both cheaper and more accurate.
4 min
Resumable LLM Streaming: How to Survive a Refresh Without Repaying for the Answer
SSE hands you a Last-Event-ID header that looks like free stream resumption. It isn't — it's a cursor with nothing behind it. The real fix is the one decision everything else follows from.
6 min
Request Hedging for LLM Tail Latency: Race the Slow Call, Don't Retry It
Every other latency fix speeds up the typical request. Hedging is the only one that attacks the slow tail — by firing a duplicate after your p95 and taking whichever finishes first.
5 min
RAFT vs RAG vs Fine-Tuning: When to Train on the Documents You Retrieve
RAG gives the model an open book; fine-tuning makes it memorize. RAFT does the thing neither does — it trains the model on bad retrieval, so it survives the wrong chunk your production retriever will hand it.
4 min
OpenTelemetry GenAI Semantic Conventions: The Spec Your Observability Tool Depends On Is Still 'Development'
Every LLM-tracing vendor now sells the same promise — open, portable, OTel-native. The schema that makes that true isn't finished, and there's an env var to prove it.
4 min
MMR vs Reranking in RAG: Why Your Top-K Returns the Same Fact Five Times
A reranker and a diversity step look like the same 'advanced RAG' upgrade. They fix opposite failures — and the benchmark that everyone cites quietly shows that turning on diversity often does nothing at all.
5 min
Mixture of Agents vs a Single Model: Why Ensembling LLMs Usually Loses to Sampling One Good Model Twice
Mixture-of-Agents wins by quality, not by variety — and a careful 2025 replication found that aggregating repeated samples from your single best model beats mixing different ones in most cases. Here's when an ensemble actually pays, and when it just adds latency.
5 min
MCP Tasks: How Long-Running Agent Work Survives a Stateless Server
The 2026-07-28 spec made MCP stateless. Long-running work and statelessness are in direct tension — and the Tasks extension resolves it by handing the bookkeeping to the client. The tell is what got deleted.
5 min
MCP Extensions, Explained: How the 2026 Spec Grows Without Breaking the Core
The next Model Context Protocol release stops adding features to the core and starts subtracting them. The Extensions framework is how — and 'in the spec' no longer means 'in the core.'
5 min
Implicit vs Explicit Prompt Caching: When to Pay for a Cache You Control
Both kinds of cache hit read at the same discount, so cost-per-hit is the wrong thing to choose on. The real split is a guarantee you pay for versus a freebie you can't shape.
5 min
Streaming Structured Output From an LLM: How to Render JSON Before It's Done
A JSON object isn't valid until its closing brace — but your UI shouldn't wait for it. The trick is realizing a streamed object is a view, not a value, and validating it exactly once: at the end.
5 min
How to Set a Timeout for an AI Agent: A Per-Call Timeout Won't Bound the Loop
The SDK's 10-minute default times out one call; an agent makes dozens. You need a deadline the whole loop shares — and cancelling to enforce it still costs tokens and can corrupt state.
6 min
How to Roll Back an AI Agent's Actions: The Saga Pattern for Tools That Can't Undo
An agent has no ROLLBACK: when step three fails, the first two already happened in the world. The fix is a compensating undo for every tool — and putting the one you can't undo last.
4 min