How to Set the Prefill-to-Decode GPU Ratio for Disaggregated Inference
Once prefill and decode live on separate GPU pools, you have to decide how many of each. The number isn't a property of your model — it's a property of your traffic, and it drifts.
4 min
Pinecone Added Full-Text Search: One Index for BM25 and Vectors Doesn't Mean One Query
Text, dense, and sparse now live in a single Pinecone index. But a search request ranks by exactly one score, so 'true hybrid' fusion quietly moves back into your code.
4 min
OpenTelemetry Catches 6 of Your Agent's 14 Failure Modes: The Five Spans It's Missing
A new benchmark maps the ways agents fail to the spans that would catch them. The GenAI conventions instrument the LLM call and the tool call — and go blind on planning, reasoning, guardrails, delegation, and memory.
5 min
Why Prefix Caching Quietly Fails in Agent Loops — and What Non-Prefix KV Reuse Does Instead
The universal advice is 'front-load your static system prompt so it gets prefix-cached.' In a tool-using or RAG agent, one mid-context insertion throws that whole cache away. CacheBlend keeps it anyway.
5 min
NIXL vs Mooncake: Choosing a KV-Cache Transfer Backend for Disaggregated Inference
Once you split prefill and decode onto separate GPUs, something has to ferry gigabytes of KV cache between them. NIXL and Mooncake are the two names you'll meet — and they aren't actually competitors.
4 min
MemoryArena vs LoCoMo: Why Agent Memory Scores 95% on the Benchmark and ~50% When It Has to Act
The agent-memory leaderboard is fought on LoCoMo, a passive-recall test. MemoryArena couples memory to action — and the same near-perfect systems fall 40 points. The gap isn't inflation; it's the wrong exam.
5 min
The Tool Bill: Why Agent Cost Tracking Is Moving to the MCP Gateway
LiteLLM v1.91.0 quietly started rolling MCP tool-call spend into the same user counters that meter tokens. It's a small line in the changelog and a large move on the board — the half of the agent bill token meters never saw.
5 min
LlamaIndex Workflows 1.0: The Orchestration Engine Left the RAG Framework Behind
The headline reads like a version bump. It isn't. Workflows 1.0 is the moment LlamaIndex's event-driven engine became a package you can install with no LlamaIndex in its dependency tree — and that changes what "using LlamaIndex" means.
4 min
LangGraph Node Error Handlers: Saga Compensation, and Why It Isn't a Timeout
LangGraph 1.2 gives a node three ways to fail — timeout, error_handler, drain. They look similar and do opposite things to your state. Mixing them up corrupts compensation.
4 min
Langflow Is the First AI-Agent Platform in CISA's KEV — and Both Ways In Reach the Same Loot
An unauthenticated RCE and an authenticated cross-tenant IDOR are opposite bug classes. In Langflow they end the same way: a prompt that says 'leak api keys.'
4 min
Git for Your Corpus: LanceDB Branching Makes RAG Evals Reproducible
LanceDB 0.34.0 added table branches — writes on a branch don't touch main. The headline feature is substring search; the sleeper is that the hard part of RAG evals was never the metric. It was holding the corpus still.
4 min
LanceDB's FM-Index: Substring Search for Code, Logs, and IDs — Not Word Search
Full-text search tokenizes your text into words, so it structurally cannot match a fragment inside a token. LanceDB's new FM-Index indexes the raw bytes instead — the exact-match primitive code and log agents were missing.
4 min
Kubernetes Has No Word for "One Agent": Inside the Sandbox CRD and Its Warm Pool
Deployments assume fungible replicas; StatefulSets assume a numbered set. An AI agent session is neither — it's a singleton with a stable identity, one of a million uniques. The kubernetes-sigs Agent Sandbox project adds the primitive that was missing, plus a warm pool that hands one over in milliseconds.
4 min
JadePuffer: The First Agentic Ransomware Ran on the Same Harness Tricks You Do
Sysdig documented an AI agent that ran a ransomware operation end to end. The scary part isn't the model — it's that the attacker's reliability engineering was indistinguishable from yours.
5 min
How to Throttle an Agent Against a Third-Party API Rate Limit
The instinct is to rate-limit per user. An agent breaks that in one move: a single user's run fans out into hundreds of calls, and the ceiling that binds isn't yours — it's the API you're calling.
5 min
How to Resume an Agent's Stream After the Connection Drops
Durable execution saves the agent's work when the server dies. It does nothing for the user whose phone dropped Wi-Fi mid-answer — that's a different resume problem, on the other side of the wire, and the new stateless MCP spec quietly made it harder.
4 min
How to Build a Coding Agent (The Loop Is the Easy Part)
A working coding agent is a few hundred lines and four tools — a weekend. What separates a toy from Claude Code is everything that isn't the loop: the edit contract, what you keep out of context, and whether it runs the tests.
4 min
Grok 4.5 vs Opus 4.8: Losing the Benchmark, Winning the Token Bill
On xAI's own SWE-Bench Pro numbers, Grok 4.5 loses to Opus 4.8 by 4.5 points — and finishes the same task for roughly a seventeenth of the output cost. The interesting number isn't the price. It's the token count.
5 min
Go AI Agent Frameworks: Eino vs LangChainGo vs Genkit (and When to Skip the Framework)
In Python, an agent framework sells you concurrency, cancellation, and retries. Go ships all three in the standard library — so the real question in Go isn't which framework, it's whether you need one.
5 min
Why DSPy Rebuilt ReAct: The Trajectory String Was Quietly Breaking Prompt Caching
DSPy's ReActV2 looks like a native-tool-calling upgrade. The real fix is deeper — the classic ReAct loop re-serialized its whole scratchpad into one prompt every turn, which silently defeated provider prompt caching. Moving to structured history cut cost up to 50%.
4 min
You're Measuring Your Semantic Cache Wrong: Hit Rate Hides the False Positives
Everyone reports the hit rate. The number that decides whether a semantic cache is safe to ship is the false-positive rate — and the fix for false positives eats the exact win you installed the cache to get.
4 min
LangChain's Deep Agents Now Ships Its Own Coding Agent — and Speaks ACP
In early July, Deep Agents quietly split into three shippable packages: a model-agnostic harness, a terminal coding agent, and an ACP adapter. The library became a product line — and unbundled the coding agent from both the model and the editor.
4 min
Decoder-Backbone Rerankers: Why Your Cross-Encoder Is Now an LLM (and Fails Like One)
The word 'cross-encoder' still means one query-doc pair, one relevance score. But the model underneath quietly flipped from a BERT encoder to a causal decoder — and it brought the LLM's failure modes with it.
4 min
CrewAI Conversational Flows: What 'Chat' Actually Adds to a Crew
CrewAI 1.15 shipped conversational flows, and it's easy to read that as "your crew can hold a conversation now." It can't. What shipped is a persisted, resumable flow behind a poll loop — and that distinction decides how you build.
4 min
The Claude Memory Tool Ships No Storage — It's a Contract You Implement
Anthropic's memory tool gives Claude a /memories directory it can read and write across sessions. But the directory is a fiction, the store is your code, and so is every line of the security.
5 min
Claude Code Dynamic Workflows vs Subagents: When to Move the Plan Into Code
Subagents let Claude delegate a few tasks per turn. Dynamic workflows fan out hundreds. The line between them isn't how many agents you need — it's whether the plan is stable enough to freeze into a script.
4 min
Chinese AI Models Passed 45% of OpenRouter's Tokens. US Labs Still Take Most of the Money.
The token-share charts everyone is quoting measure the wrong thing. On the same marketplace where Chinese open-weight models now move most of the tokens, Anthropic — with roughly an eighth of the volume — still captures nearly half the revenue. That gap is the whole story.
5 min
Webhooks vs Polling for Long-Running Agent Tasks: Why Agents Reversed the Default
For a decade the advice was "stop polling, use webhooks." The agent runtime quietly broke the webhook's core assumption — so the newest async surfaces ship polling first.
5 min
How to Scale a Vector Database to Billions of Vectors
Sharding vectors is nothing like sharding rows. The real decision isn't where the data lives — it's how many shards each query is allowed to skip, and what recall you pay to skip them.
4 min
Tracing MCP Tool Calls Without Sessions: Why traceparent Became the Correlation ID
MCP's 2026-07-28 spec deletes the session handshake that ops teams quietly used to stitch an agent's tool calls together in their logs. The replacement is W3C Trace Context — and it doesn't do the same job.
5 min