Contents
The "Parallel Equals Cheap" Misconception
A staple technique in agent architecture is the "fan-out": one orchestrator spawns several sub-agents in parallel, each assigned an independent subtask. Research, multi-service data gathering, cross-cutting search — it's true that parallelization dramatically shrinks wall-clock time.
But here's the trap many teams fall into: mistaking "parallel makes it faster" for "parallel makes it cheaper." Token billing is driven not by execution time but by input/output token volume. Whether you hit 10 endpoints sequentially or in parallel, if the real API tokens are the same, billing looks unchanged. In reality, parallelization creates three hidden costs.
Fan-out is a trade where you pay tokens to buy latency. Parallel is justified only when the value of wall-clock savings exceeds three costs: (1) context duplication, (2) waiting due to gating, and (3) retry multiplication. We visualize that break-even point with KanseiLink's measured latency and early success-rate data.
Cost #1: The Context-Duplication Tax
Each sub-agent carries its own independent context: system prompt, tool definitions (MCP tool schemas), task instructions, a snapshot of shared state — and these are duplicated once per sub-agent. If the orchestrator fans out to 5 sub-agents, the shared system prompt and tool definitions are billed as input tokens 5 times.
MCP tool definitions are especially heavy. The fact that tool definitions occupy a substantial fraction of context is a known problem, and it gets multiplied by the number of sub-agents. Even where sequential execution could reuse a single context, parallel execution has each agent carry the full set. This is the "context-duplication tax."
"More sub-agents means faster" holds up to a point, but input tokens grow linearly with fan-out width. Without reusing the shared portion via prompt caching, 5-way parallel effectively bills input tokens 5x. There is a regime where speed plateaus but cost keeps climbing.
Cost #2: Gating by the Slowest Service
A parallel batch typically waits for all sub-agents to finish before moving on. So the batch's duration is set by the slowest service's latency. In KanseiLink's measured data, latency differences across services are not negligible.
| Service | Avg latency | Success | Role in a parallel batch |
|---|---|---|---|
| Backlog MCP | 128ms | observing | Fast |
| Slack MCP | 163ms | observing | Fast |
| kintone MCP | 199ms | observing | Mid-speed |
| freee MCP | 216ms | observing | Mid-speed |
| SmartHR | 337ms | observing | Slow |
| Salesforce Japan | 474ms | observing | Slowest — the gate |
Here's the crux. Hit only Backlog (128ms) and Slack (163ms) in parallel and the batch finishes in ~163ms. But the moment you mix one Salesforce call (474ms) into the same batch, the whole batch waits 474ms. The fast-service sub-agents sit idle waiting for the slowest one to finish. Raise the parallelism all you like — if the gating step is slow, the wall-clock benefit plateaus. This is Amdahl's Law, the agent edition.
Cost #3: Retry Multiplication on Low-Success Services
The third is the most overlooked. Failures generate retries, and retries are additional runs that re-consume context. The wider the fan-out, the larger the absolute number of failures on low-success services.
Suppose you fire 10 parallel calls at a service whose success rate languishes at 39%. You expect ~4 successes, leaving ~6 as retry candidates. Each retry re-consumes a sub-agent's context (duplication tax included). By contrast, fire the same 10 at a service in the 90%+ band and you get ~1 failure. At the same fan-out width, retry cost differs 6x by success-rate band.
Expected failures at 10-way parallel, by success band (illustrative assumptions)
(assuming 90%+)
(assuming 79%)
(assuming 39%)
In other words, fanning out to a low-success service doesn't just "fail fast" — it "mass-produces failures, duplication tax included." It's the economics of the retry tax, amplified by parallelization.
Break-Even — When to Go Parallel
So when is fan-out justified? The conditions where parallel wins boil down to three.
| Condition | Parallel wins | Sequential wins |
|---|---|---|
| Task dependency | Mutually independent | Prior result feeds next input |
| Service latency | High (big value in cutting wait) | Low (verified tier 128–216ms) |
| Service success rate | High (low retry-multiplication risk) | Low (services with struggling success rates) |
| Fan-out width | Many (savings beat duplication tax) | Few (duplication tax wins) |
Concretely, a high-latency research task gathering data independently from 10 services is a textbook win for parallel. If wait time dominates, there are no dependencies, and the targets are high-success, it's worth paying the duplication tax to shrink the wall clock.
Conversely, a lightweight task hitting only 2–3 verified-tier services is often cheaper sequential. Hit Backlog at 128ms x 3 sequentially and you're at ~400ms with a single context. Go 3-way parallel and speed ticks up slightly while input tokens triple. A textbook case of "faster, but not worth it."
The Practical Playbook
- Parallelize only independent tasks — force-parallelizing dependent tasks just reintroduces a join, leaving only the duplication tax.
- Group batches by success-rate band — don't mix verified and low-success services in one batch. Mixing gets gated by the slowest and lowest-success member, the most inefficient combination.
- Isolate slow services — Salesforce (474ms) and SmartHR (337ms) are the gating step. Split them into a separate batch so they don't stall fast-service slots.
- Reuse shared context via prompt caching — caching the system prompt and tool definitions recovers most of the duplication tax. Prompt-caching cost savings pair especially well with fan-out.
- Tune fan-out width to success rate — narrow the width for low-success services to cap the absolute number of retries. Check latency and success rate with
get_insightsbeforehand and feed it into batch design.
Fan-out is powerful, but not free. Treat it as "a trade buying latency with tokens": know the target services' latency and success rate up front and design batches accordingly, and you minimize all three costs — duplication tax, gating, and retry multiplication. KanseiLink's measured data is the input to that batch design.
FAQ
Does running sub-agents in parallel increase cost?
In most cases, yes. Parallel (fan-out) cuts wall-clock time but does not lower token cost, and creates three hidden costs: (1) context duplication per sub-agent, (2) gating by the slowest service, and (3) retry multiplication on low-success services. Fast does not mean cheap.
Where is the break-even point for fan-out?
When the value of wall-clock savings exceeds the cost of duplicated tokens. Parallel wins more as three conditions align: (1) tasks are independent, (2) target services are high-latency, (3) target services are high-success. Hitting only a few cheap, fast, high-success verified services is more efficient sequentially.
Why does the slowest service gate the whole batch?
Because a parallel batch waits for all sub-agents to finish. The verified tier runs 128–216ms, but Salesforce is 474ms and SmartHR 337ms. Mix in one Salesforce call and the whole batch waits 474ms even if everything else is 128ms — the slow service becomes the rate-limiting step.
Why does hitting low-success services in parallel inflate cost?
Failures generate retries, and retries are additional runs with the duplication tax included. For example, fire 10 parallel calls at a service with a 39% success rate and you expect ~6 failures; in the 90%+ band, ~1. At the same fan-out width, retry cost differs 6x by success-rate band.
What are best practices for controlling parallel-execution cost?
(1) Parallelize only independent tasks, (2) group batches by success-rate band, (3) isolate slow services into a separate batch, (4) reuse shared context via prompt caching, (5) tune fan-out width to success rate. Use get_insights to check latency and success rate up front and feed it into batch design.
The latencies here are KanseiLink connection-probe measurements (as of May 2026): Backlog 128ms, Slack 163ms, kintone 199ms, freee 216ms, SmartHR 337ms, Salesforce Japan 474ms. Per-service measured success rates are still accumulating at KanseiLink; the success bands in this article (90%+/79%/39%) are illustrative assumptions for the expected-value math, not measured values for any specific service. "Expected failures" are expected values derived from success rate (e.g. 39% success x 10 calls → ~6.1 expected failures); actual distributions vary per trial. The cost structure of context duplication, gating, and retry multiplication is a model based on a common orchestrator/sub-agent architecture (each sub-agent holds an independent context and the batch joins on completion); it varies with runtime implementation (context sharing, streaming aggregation, partial-completion handling). Concrete token prices and billed amounts depend on the model, provider, and prompt-cache usage, so we discuss relative cost structure rather than naming amounts. The break-even assessment is an analytical model; we recommend empirical validation on your own workload. Check each get_insights for the latest latency and success rate.