RAG

What is RAG?

A model's knowledge is frozen at training time and blind to your private or recent data. Retrieval-augmented generation fixes that without retraining: find the relevant documents, put them in the prompt, and let the model answer grounded in real, current text.

01The answer, then the intuition

Open-book, not closed-book

Ask a bare model about your company's refund policy and it will guess — confidently, and often wrong — because that fact was never in its training data. It's taking a closed-book exam on material it never studied. RAG turns it into an open-book exam: before answering, it looks up the relevant page and reads from it.

The lookup is by meaning, not keywords. The question and every document are turned into embeddings, and the closest chunks by cosine similarity are pulled into the prompt. Toggle retrieval on and off and watch the same model go from a plausible fabrication to a grounded, cited fact:

RAG — grounding a frozen model, live

A tiny vector store of company facts. Toggle retrieval and compare the answers.

Question: "What's our refund window?"

Vector store · ranked by similarity to the question

Answer

02Mechanics

The pipeline, end to end

Index (once, offline). Split your documents into chunks, turn each into an embedding vector, and store them in a vector database (next chapters). This is the "library" the model will consult.
Retrieve (per query). Embed the user's question with the same model, then find the chunks whose vectors are nearest — semantic search. Take the top-$k$ (here, top-2). Keyword search would miss "refund window" ↔ "refunds within 30 days"; embeddings match on meaning.
Augment. Paste the retrieved chunks into the prompt as context, with an instruction like "answer using only the sources below, and cite them."
Generate. The model answers conditioned on real text it can quote — so it's current, grounded, and auditable, and far less likely to hallucinate. Update the store and the answers update instantly; no retraining.

RAG's limits are retrieval's limits: if the right chunk isn't found, or the chunking is poor, the model is back to guessing. Most "RAG doesn't work" stories are really "retrieval didn't return the right thing" — which is why the vector-search chapter matters.

04The math

expand ▾

Retrieve by similarity, condition on the result

Every chunk $d_i$ and the query $q$ are embedded into the same space. Relevance is cosine similarity:

$$ \text{score}(q, d_i) = \cos(\mathbf{e}_q, \mathbf{e}_{d_i}) = \frac{\mathbf{e}_q \cdot \mathbf{e}_{d_i}}{\lVert \mathbf{e}_q\rVert\,\lVert \mathbf{e}_{d_i}\rVert} $$

Take the top-$k$ chunks $R = \text{top-}k_i\,\text{score}(q,d_i)$, and generate the answer conditioned on both the retrieved text and the question:

$$ y \sim P\big(y \mid R,\, q\big) $$

Compared with the bare $P(y\mid q)$ from the last chapter, the only change is what's in the context — retrieval injects grounded evidence. That's the whole trick: RAG is prompting where the context is fetched, by meaning, at query time.

05The code

expand ▾

Retrieval in a dozen lines

Rank a tiny store by cosine similarity to a query and take the top-2. This is the heart of every RAG system.

retrieve.py

import numpy as np

def cos(a, b):
    a, b = np.array(a, float), np.array(b, float)
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

query = [0.9, 0.1, 0.2]                          # "refund window" (toy embedding)
store = {
    "Refunds within 30 days of purchase.": [0.88, 0.12, 0.15],
    "Shipping takes 3-5 business days.":    [0.10, 0.90, 0.20],
    "Warranty covers defects for 1 year.":  [0.30, 0.20, 0.85],
    "Returns accepted in original box.":    [0.80, 0.25, 0.10],
}
ranked = sorted(store.items(), key=lambda kv: -cos(query, kv[1]))
for text, v in ranked:
    print(f"{cos(query, v):.3f}  {text}")
# 0.998  Refunds within 30 days of purchase.   <- retrieved
# 0.977  Returns accepted in original box.     <- retrieved
# 0.537  Warranty covers defects for 1 year.
# 0.256  Shipping takes 3-5 business days.

06The economics

Fresh, private, and auditable — for the price of a lookup

Grounding → money

RAG is how a frozen model becomes an enterprise product. It makes knowledge current (update the store, not the weights), private (your data never enters training), and auditable (every claim can cite a source). For most business uses that trio beats fine-tuning, because company data changes daily and someone always needs to know where an answer came from.

The cost shifts to two places: the retrieval infrastructure (a vector database and the embedding pipeline) and, on every call, the tokens of injected context — which enlarge the prompt and its KV cache. So RAG trades a modest per-query token premium for accuracy and trust, and it scales cheaply because indexing is a one-time cost amortized over every future question.

This is close to home: grounding claims in citable sources is exactly what a research desk like the Circuit is for. The same discipline that makes RAG trustworthy — show the source, quote the evidence, let the reader check — is the discipline that separates analysis worth paying for from confident noise.

07Going deeper

expand ▾

The primary sources

Lewis et al. (2020) — Retrieval-Augmented Generation · the original RAG paper.
Karpukhin et al. (2020) — Dense Passage Retrieval · embedding-based retrieval.
Gao et al. (2023) — RAG for LLMs: A Survey · the modern design space.
Brown et al. (2020) — GPT-3 · why in-context conditioning works at all.

"First Principles — how AI actually works", chapter "What is RAG?", by The Catch. CC-BY 4.0. Hosted at TheCatch.AI / Tech Stack / AI Principles.

Vector Search

Vector search

Under every RAG system is one hard problem: find the handful of vectors nearest a query, among millions or billions. Comparing them all is too slow — so approximate nearest-neighbor search trades a sliver of accuracy for enormous speed.

01The answer, then the intuition

Don't check everything — check the right neighborhood

An embedding turns every document into a point in high-dimensional space, where nearness means similar meaning. Retrieval is then just geometry: find the points closest to the query point. The naive way — measure the distance to every stored vector and sort — is exact but costs a full scan per query. At a billion vectors, that's hopeless.

The insight of approximate nearest-neighbor (ANN) search is that you don't need to check everything. Pre-build an index that lets you jump to the right neighborhood and examine only a few candidates. You give up a tiny chance of missing a true neighbor in exchange for searching orders of magnitude fewer points. Move the query below and watch exact scan versus approximate:

Nearest-neighbor search — exact vs approximate

Click or drag anywhere to move the query. The 3 nearest points light up; examined points are ringed.

—

points compared

—

correct top-3 found

02Mechanics

How you skip the full scan

The problem is scale. Exact search compares the query to all $N$ vectors, each of dimension $d$ — fine for thousands, ruinous for billions. Every RAG query would otherwise touch the entire store.
HNSW graphs. The dominant method builds a navigable "small-world" graph linking each vector to nearby ones, with a few long-range links layered on top. Search starts anywhere and greedily hops toward the query, reaching the neighborhood in roughly logarithmic steps — examining a few dozen points instead of billions.
IVF & quantization. Alternatives cluster the space into cells and search only the nearest cells (IVF), and product quantization compresses each vector into a few bytes so more fit in memory. These cut both time and storage, at some cost to precision.
The trade-off: recall. ANN can occasionally miss a true neighbor — measured as recall@k. Turning a knob (examine more candidates) raises recall toward 100% but costs speed. Vector databases — pgvector, FAISS, Qdrant, Pinecone, Weaviate — are largely engines for tuning this recall-speed-memory triangle.

So vector search is the unglamorous machinery that makes semantic retrieval possible at real scale — the reason "search by meaning over everything you own" is a product and not a research demo.

04The math

expand ▾

From linear to logarithmic

The $k$ nearest neighbors of a query $q$ minimize distance over the store:

$$ \text{kNN}(q) = \operatorname*{arg\,min}_{|S|=k}\; \sum_{d_i \in S} \lVert q - d_i \rVert $$

Exact search evaluates that distance for every vector — cost $O(N \cdot d)$, linear in the number of documents. An HNSW index changes the exponent: greedy graph traversal reaches the query's neighborhood in roughly

$$ O\big(d \cdot \log N\big) \quad\text{expected, per query} $$

The difference between $N$ and $\log N$ is the whole game: at a billion vectors, $\log_2 N \approx 30$, so ANN examines on the order of tens of candidates instead of a billion. The price is recall@k $= \frac{|\text{retrieved} \cap \text{true top-}k|}{k}$, which a good index keeps at 95–99% while searching a vanishing fraction of the data.

05The code

expand ▾

Exact search, and why it doesn't scale

Exact k-NN is a full scan. The comparison count is the whole reason ANN indexes exist.

knn.py

import numpy as np

q = np.array([5.0, 5.0])
store = {"A":[5.2,4.8], "B":[1.0,1.0], "C":[6.0,5.5], "D":[8.5,2.0],
         "E":[4.6,5.3], "F":[9.0,9.0], "G":[2.0,7.0], "H":[5.1,6.2]}

comparisons = 0
dists = {}
for name, p in store.items():
    comparisons += 1                       # exact = compare to EVERY vector
    dists[name] = np.linalg.norm(q - np.array(p))

top3 = sorted(dists, key=dists.get)[:3]
print(f"exact scan: {comparisons} comparisons")     # 8 (here) ... N at scale
print("3 nearest:", [(n, round(float(dists[n]), 2)) for n in top3])
# exact scan: 8 comparisons
# 3 nearest: [('A', 0.28), ('E', 0.5), ('C', 1.12)]
# an HNSW index would touch ~log(N) candidates and return the same 3

06The economics

The plumbing that makes AI memory affordable

Search → money

Vector search is why AI can have a usable memory at all. Without ANN, grounding a model in a large corpus would mean scanning every document on every query — computationally and financially impossible at scale. ANN turns that linear cost into a logarithmic one, and that single change is what makes retrieval cheap enough to put under every product. The vector database has become its own infrastructure category precisely because this problem is universal.

The economics live in the recall-speed-memory triangle. Tighter recall costs more compute and memory per query; looser recall is cheaper but risks missing the right chunk and sending the model back to guessing. Tuning that trade-off is a real operating decision, because retrieval quality caps the quality of everything built on top of it.

For the Circuit, this is the demand side's foundation: the same embeddings that opened this book close the loop here, as the searchable memory that lets AI reason over a company's — or a researcher's — entire corpus. It's the least visible layer, and one of the most load-bearing.

Part IV complete

You can now build with a model, not just describe one

From prompting a frozen model, through grounding it with retrieval, giving it tools to act, choosing among adaptation strategies, measuring quality with evals, and the vector search underneath it all — you've gone from theory to a working mental model of an AI application. Part I built the machine, Part II the model, Part III the cost of running it, and Part IV the craft of using it.

That's 25 chapters and four complete parts — the mechanics, end to end. What remains is the part only this desk is built to tell: where these mechanics meet the money. See the full curriculum →

07Going deeper

expand ▾

The primary sources

Malkov & Yashunin (2016) — HNSW · the navigable small-world graph index.
Johnson et al. (2017) — Billion-scale similarity search (FAISS) · ANN at scale.
Jégou et al. (2011) — Product Quantization · compressing vectors for search.
ANN-Benchmarks · the recall-vs-speed trade-off, measured across libraries.

"First Principles — how AI actually works", chapter "Vector search", by The Catch. CC-BY 4.0. Hosted at TheCatch.AI / Tech Stack / AI Principles.

Context Window

The context window

The context window is a model's working memory — the number of tokens it can hold and attend to at once. Everything inside it the model can use; everything outside it, it never sees. And making it bigger is brutally expensive.

01The answer, then the intuition

How much the model can hold at once

A model doesn't have long-term memory between turns; within a single request it has the context window — the maximum number of tokens it can read and reason over together. A whole conversation, a pasted document, the system instructions: it all has to fit. Run past the limit and the oldest tokens fall off the edge, or get truncated, and the model simply doesn't know they existed.

Windows have grown fast — from ~2,000 tokens in early GPT-3 to 128,000 in GPT-4-class models to over a million in the largest. But that growth fights a hard wall you've already met: attention's cost grows with the square of the window, and the model must cache every token's keys and values in fast memory. Drag the window and watch what it costs (on a 70B-class model):

The cost of context — drag it

KV cache and attention work for one request on an 80-layer, 8192-wide model in fp16.

—

1K32K1M

—

KV cache memory

—

80 GB GPUs of cache

—

attention work vs 1K

02Mechanics

Why it's capped, and how it's stretched

Two costs set the ceiling, both from attention:

Compute grows quadratically. Every token attends to every other, so the work scales as $n^2$. Going from a 1K window to a 128K one is a 128× increase in length — and therefore about 16,000× the attention work (128 squared). It explodes.
Memory grows linearly — and ruinously. To avoid recomputing, the model stores every token's keys and values, the KV cache. That cache grows with the window and the model's depth and width, and it lives in scarce high-bandwidth memory.

So engineers fight on three fronts: position tricks like RoPE and ALiBi that let a model generalize to lengths it wasn't trained on; efficient attention like FlashAttention, sliding-window, and sparse patterns that cut the constants (or even the exponent); and simply more memory. One caveat worth knowing: even when a model can read a million tokens, it often attends less to the middle — the "lost in the middle" effect — so a bigger window isn't automatically better understanding.

04The math

expand ▾

Quadratic compute, linear cache

For a window of $n$ tokens, width $d$, and $N$ layers, the two costs are:

$$ \text{attention compute} \;\propto\; n^2 d \qquad\qquad \text{KV cache bytes} \;=\; 2 \, N \, n \, d \, b $$

The $2$ counts keys and values, $b$ is the bytes per number ($2$ in fp16). The compute term is the famous quadratic: doubling $n$ quadruples the attention work. The cache term is "only" linear in $n$ — but the constant is enormous, because it also multiplies by every layer and the full width.

Make it concrete. For an 80-layer, 8192-wide model, each token costs $2 \cdot 80 \cdot 8192 \cdot 2 \approx 2.6$ MB of cache. So a 128K-token window holds about 340 GB of KV cache — roughly five 80 GB GPUs, before the model produces a single word. A one-million-token window is measured in terabytes. (These are full multi-head-attention figures — the honest worst case. Real 70B-class models use grouped-query attention, sharing keys and values across heads to shrink the cache by roughly 8×; it is precisely the trick invented to fight this wall.) That is why long context is a hardware problem, not a software setting.

05The code

expand ▾

The KV-cache calculator

The exact function behind the slider — runnable.

kv_cache.py

def kv_cache_gb(n_tokens, n_layers, d_model, bytes_per=2):
    # 2 = one key + one value vector per token, per layer
    return 2 * n_layers * n_tokens * d_model * bytes_per / 1e9

# an 80-layer, 8192-wide model in fp16
for ctx in [4096, 32768, 131072, 1_000_000]:
    print(f"{ctx:>9,} tokens -> {kv_cache_gb(ctx, 80, 8192):8.1f} GB of KV cache")
# ->     4,096 tokens ->     10.7 GB of KV cache
#       32,768 tokens ->     85.9 GB of KV cache
#      131,072 tokens ->    343.6 GB of KV cache
#    1,000,000 tokens ->   2621.4 GB of KV cache

06The economics

Where the square meets the customer

Window → money

The context window is where every cost in this Part comes due at once. The tokens set $n$; attention squares it for compute; the KV cache turns it into hundreds of gigabytes of high-bandwidth memory per long request. This is the most direct reason inference is memory-bound, and why providers charge more for long-context calls — you are renting scarce HBM by the token-squared.

It also reframes the build-out. A huge share of data-center spending isn't about training ever-larger models; it's about serving them — holding millions of users' KV caches in memory at once. Every leap in advertised context length ripples straight into demand for the exact resource the Circuit calls the memory wall.

So the context window is the perfect closing note for the foundations: it is the single setting where the token, the embedding, attention, the network, the transformer, and the parameter count all collapse into one number you pay for. Read the spec "1M token context," and now you know precisely what it costs to keep that promise.

That completes Part I — the Foundations. From a token to the full transformer and the economics of running it, you now have the whole machine in view. Part II opens the model itself: how LLMs are trained, fine-tuned, and how they differ. Back to the curriculum →

07Going deeper

expand ▾

The primary sources

Su et al. (2021) — RoFormer (RoPE) · rotary position encodings that help models extend their length.
Dao et al. (2022) — FlashAttention · making long-context attention fit in memory.
Liu et al. (2023) — Lost in the Middle · why a longer window isn't always better understanding.
Beltagy et al. (2020) — Longformer · sparse attention for long documents.

"First Principles — how AI actually works", chapter "The context window", by The Catch. CC-BY 4.0. Hosted at TheCatch.AI / Tech Stack / AI Principles.

Agents

Agents & tool use

A model on its own can only write text. Put it in a loop with tools — search, a calculator, code, an API — and it can act: take a step, read the result, decide the next step, and repeat until the task is done. That loop is an agent.

01The answer, then the intuition

Think, act, observe — then think again

A bare model can't check today's price, run a calculation exactly, or query your database — its knowledge is frozen and its only output is text. An agent gets around this by letting that text be an action. The model writes a structured tool call; the surrounding program runs it and feeds the result back; the model reads the result and decides what to do next. Loop until it has the answer.

This "think → act → observe → repeat" cycle turns a predictor into a doer. Step through one agent solving a task that needs two tools it doesn't have built in — a live lookup and exact arithmetic:

An agent loop, one step at a time

The model can’t know a live price or do exact math alone — so it calls tools and reads the results.

Task: "What's a 15% tip on the current price of Bitcoin?"

02Mechanics

What actually makes a model an agent

Tools with schemas. Each tool is described to the model — its name, what it does, and the shape of its arguments. The model is trained to emit a structured call (e.g. JSON) when it wants one, which the program can parse and execute.
The loop. The core is dead simple: send the context to the model; if it returns a tool call, run the tool and append the result as an observation; if it returns an answer, stop. Repeat. Each turn the context grows with the full history of thoughts, actions, and observations — this is the ReAct pattern (reason + act).
Why tools matter. They patch the model's weaknesses precisely: a calculator for exact math it's bad at, search for current facts, code execution for real computation, database queries for private data. The model supplies the judgment about which tool to use when; the tools supply the ground truth.
The hard part: reliability. More steps means more chances to go wrong — a bad tool call, a misread result, or an infinite loop. Real agents need guardrails: step limits, validation, retries, and human checkpoints. An agent that's right 95% per step is only ~60% right after ten steps.

So an agent isn't a smarter model — it's the same model wrapped in a control loop that lets it interact with the world and correct course. The intelligence is in the model; the agency is in the loop.

04The math

expand ▾

A policy over a growing context

At each step $t$, the model acts as a policy, sampling an action $a_t$ from the full history of what it has thought, done, and seen:

$$ a_t \sim P\big(a \mid x,\; h_t\big), \qquad h_t = (a_1, o_1, \dots, a_{t-1}, o_{t-1}) $$

If $a_t$ is a tool call, the environment returns an observation $o_t = \text{tool}(a_t)$, which is appended to the history; if $a_t$ is an answer, the loop halts. Each step is a full forward pass over an ever-growing context — so a $k$-step task costs on the order of:

$$ \text{cost} \approx \sum_{t=1}^{k} 2N \cdot \lvert x + h_t \rvert \;\; \Rightarrow \;\; \text{grows faster than linearly in } k $$

And reliability compounds the wrong way: if each step succeeds with probability $p$, a $k$-step chain succeeds with only $p^{k}$. At $p=0.95$ and $k=10$, that's $0.95^{10} \approx 0.60$ — which is why long agent runs are fragile and why bounding $k$ matters as much as raising $p$.

05The code

expand ▾

The whole loop, in twenty lines

A minimal agent: tools, a policy (scripted here in place of the model), and the loop that ties them together.

agent.py

def search(q):  return "$67,000" if "BTC" in q else "unknown"
def calc(expr): return eval(expr, {"__builtins__": {}}, {})
TOOLS = {"search": search, "calc": calc}

# a scripted policy standing in for the model's decisions
script = [
    ("think",  "I need the current BTC price."),
    ("act",    ("search", "BTC price")),
    ("think",  "Now compute a 15% tip on 67000."),
    ("act",    ("calc", "67000 * 0.15")),
    ("answer", "15% of $67,000 is ${}."),
]

obs = None
for kind, payload in script:                 # the agent loop
    if kind == "act":
        tool, arg = payload
        obs = TOOLS[tool](arg)               # run the tool, observe result
        print(f"ACT   {tool}({arg!r}) -> {obs}")
    elif kind == "think":
        print(f"THINK {payload}")
    else:
        print("ANSWER", payload.format(obs))
# THINK I need the current BTC price.
# ACT   search('BTC price') -> $67,000
# THINK Now compute a 15% tip on 67000.
# ACT   calc('67000 * 0.15') -> 10050.0
# ANSWER 15% of $67,000 is $10050.0.

06The economics

The expensive bet the whole build-out rests on

Agency → money

Agents are where AI stops answering questions and starts doing work — and that's the entire economic thesis of the build-out. A chatbot sells tokens; an agent that can complete a multi-step task competes with labor, a far larger market. This is the demand that the hundreds of billions in compute are betting will arrive.

But the cost structure is unforgiving. Every step is another model call over a longer context, so a single agent task can cost 10–100× a single chat reply — and the reliability math ($p^k$) means longer tasks fail more often, forcing retries that cost even more. The value has to clear a bill that grows with both the length and the fragility of the task.

That tension is the crux of the Circuit's central question. If agents become reliable enough to automate real knowledge work, the demand easily justifies the clusters being built. If they stay just unreliable enough to need a human watching, the economics stay stubbornly hard. The whole payoff of the build-out rides on which way that goes — which is exactly what an honest research desk should be measuring, not assuming.

07Going deeper

expand ▾

The primary sources

Yao et al. (2022) — ReAct · interleaving reasoning and tool actions.
Schick et al. (2023) — Toolformer · teaching models to call tools.
Model Context Protocol (MCP) · an open standard for connecting tools to models.
Anthropic — Building Effective Agents · patterns and the reliability problem.

"First Principles — how AI actually works", chapter "Agents & tool use", by The Catch. CC-BY 4.0. Hosted at TheCatch.AI / Tech Stack / AI Principles.

Multi Agent

Multi-agent & what comes next

The frontier isn't a bigger model — it's many agents coordinating: an orchestrator delegating to workers, agents calling agents, whole workflows automated. The promise is enormous. So is the problem: reliability compounds the wrong way.

01The answer, then the intuition

The whole rides on the weakest step

A single agent can do a task; a multi-agent system decomposes a big job across many — a planner splits the work, specialist workers run in parallel, a synthesizer merges the results. Done well, that's how AI moves from answering questions to completing projects. It's the shape most people mean by "agentic AI."

But chaining steps multiplies their failure rates. If each step is 90% reliable, ten steps in a row succeed only $0.9^{10} \approx 35\%$ of the time — the chain is far less reliable than any part of it. Drag the length of the workflow and watch success collapse, then turn on verification and watch it come back:

The reliability wall — and how verification beats it

A workflow of n sequential agent steps, each reliable with probability p. Overall success is pⁿ.

—

overall workflow success

workflow length (steps)10

per-step reliability90%

02Mechanics

How you build a system that survives its own length

Orchestration patterns. The common shapes: an orchestrator-worker split (a lead agent plans and delegates), parallel fan-out (independent subtasks run at once, then merge), and pipelines (each stage feeds the next). Parallelism is powerful because it keeps the dependent chain short — and it's the short chain that reliability depends on.
Verification is the fix. The escape from $p^n$ is to raise $p$ at each step. Have several agents independently check a step and take the majority; a step that's 90% reliable becomes ~99.9% with three votes. Adversarial verification — agents trying to refute a result — catches plausible-but-wrong outputs a single pass misses.
Bounded autonomy. Real systems cap the number of steps, validate tool outputs, and insert human checkpoints at high-stakes moments. The art is spending verification where errors are costly and letting cheap steps run free.
What comes next. The open frontiers: continual learning (models that update from experience), agent-to-agent economies, better long-horizon planning, and — underneath all of it — reliability that holds over hundreds of steps. None is solved. That's not a caveat; it's the actual state of the art.

The honest summary: multi-agent systems work impressively in demos and unevenly in production, and the gap between the two is almost entirely this reliability problem. Whoever closes it unlocks the automation the whole industry is betting on.

04The math

expand ▾

Why length is the enemy, and redundancy the cure

A workflow of $n$ sequential steps, each independently reliable with probability $p$, succeeds only if all succeed:

$$ P_{\text{success}} = p^{\,n} $$

Because $p < 1$, this decays exponentially in length — the source of the wall. Verification attacks $p$ directly. With $k$ independent checks per step, the step's effective reliability rises to $1-(1-p)^k$, so the whole chain becomes:

$$ P_{\text{verified}} = \Big(1 - (1-p)^{k}\Big)^{n} $$

The leverage is enormous. At $p=0.9$, $n=10$: the naive chain is $0.9^{10}=34.9\%$, but with $k=3$ the per-step reliability is $1-0.1^{3}=0.999$, and the chain is $0.999^{10}=99.0\%$. The cost is that each verified step now runs $k+1$ model calls — so reliability is bought with tokens. The central engineering trade of the agentic era is exactly this: how much verification to buy, and where.

05The code

expand ▾

The wall, and the way through

Naive chains collapse with length; verification restores them — at a token cost.

reliability.py

def chain(p, n):     return p ** n              # all n steps must succeed
def verified(p, k):  return 1 - (1 - p) ** k    # k independent checks per step

p = 0.90
print(f"naive 10-step chain: {chain(p, 10)*100:.1f}%")
print(f"per-step w/ 3 votes: {verified(p, 3):.3f}")
print(f"verified 10-step:    {chain(verified(p, 3), 10)*100:.1f}%")

for n in (1, 5, 10, 20):
    print(f"  n={n:>2}: naive {chain(p, n)*100:5.1f}%   "
          f"verified {chain(verified(p, 3), n)*100:5.1f}%")
# naive 10-step chain: 34.9%
# per-step w/ 3 votes: 0.999
# verified 10-step:    99.0%
#   n=20: naive 12.2%   verified 98.0%   <- length kills naive chains; verification saves them

06The economics

The bet the entire build-out is waiting on

Reliability → money

Multi-agent automation is the demand story that justifies the whole build-out. A reliable system that completes real multi-step work doesn't sell tokens — it competes with salaries, a market orders of magnitude larger than chat. If agents cross the reliability threshold for knowledge work, the revenue easily clears the capex. If they don't, the demand the spending assumes simply doesn't arrive.

The reliability math is why this is genuinely uncertain, not just hype. Verification is the known fix, but it multiplies the token cost per task — so the very thing that makes agents trustworthy also makes them expensive. Whether reliable-enough automation lands below the price of the human it replaces is an open, quantitative question, and it's the one that decides the payoff.

This is the Circuit's forward edge. Not "will AI get smarter" — the scaling law answers that — but "will agents get reliable and cheap enough, fast enough, to generate the demand the capital already assumes." That, more than any benchmark, is what an honest desk should be measuring. The book has taught the mechanics; this is where they meet the biggest open bet.

Part V complete

You've followed the mechanics all the way to the money

From the scaling law that makes the bet rational, through the labs and the supply chain that concentrate the power, to the economics of a single token and the agentic reliability wall that decides the demand — Part V is where everything the earlier parts built collides with economics. This is the layer only a research desk is built to tell.

One part remains: the practitioner's Part VI — choosing a model, optimizing cost, guardrails, and the tool landscape. The theory is done; what's left is doing it well. See the full curriculum →

07Going deeper

expand ▾

The primary sources

Anthropic — Building a Multi-Agent Research System · orchestrator-worker patterns in practice.
Wu et al. (2023) — AutoGen · a framework for multi-agent conversation.
Zaharia et al. (2024) — The Shift to Compound AI Systems · systems over single models.
Yao et al. (2022) — ReAct · the reasoning-and-acting loop agents are built on.

"First Principles — how AI actually works", chapter "Multi-agent & what comes next", by The Catch. CC-BY 4.0. Hosted at TheCatch.AI / Tech Stack / AI Principles.