Parallel Tool Calls

Goal

Identify which tool operations may safely run in parallel and join their results without violating dependencies, budgets, or side-effect rules.

Start with dependencies

Suppose you are preparing breakfast. Toasting bread and pouring juice can happen at the same time because neither result is needed to begin the other. Spreading butter on the toast cannot happen before the toast exists. "Can these jobs run at once?" is therefore really a dependency question.

Parallel tool calls work the same way. Independent operations can overlap and reduce total waiting time, but dependent or conflicting operations must stay ordered. The safe mental model is a small dependency graph: arrows mean "this result or effect must happen before that work can start." Parallelism is an optimization only after those arrows and resource limits are clear.

weather_lookup ─┐
                ├─→ combine results
calendar_read ──┘
refund_order ─────→ waits for approval first

Parallel reads still have limits

Read-only operations are often easier to parallelize, but even reads can have limits. Two expensive searches may exceed a rate limit or shared budget. A harness should bound concurrency and include all parallel work in the task's cost and timeout accounting.

Side effects require more caution. Two writes to the same record may race. A refund and a cancellation might conflict. Even if each action is individually authorized, running them concurrently can create an illegal combined state. The controller should serialize conflicting effects or use application-level concurrency controls.

Join, cancel, and trace branches

Joining parallel results means waiting for the required set of outcomes and deciding what to do with partial failure. If one of three independent reads fails, the task may continue with two results, retry the failed call, or stop because all three were required. That rule should be explicit before execution.

Cancellation also matters. If one parallel branch proves the task should stop, the harness may need to cancel or ignore other branches. Cancellation should not pretend a side effect was undone. An already-completed external action still needs reconciliation and recording.

Tracing parallel work requires parent-child relationships and timing. Two child spans can overlap under the same controller step. This makes it possible to see that total latency was closer to the slowest branch than the sum of all branch durations.

Parallelism does not add authority

Parallelism does not increase authority. Each call still passes tool selection, permission, approval, and sandbox rules. A common mistake is to validate the batch as a whole and then assume every child action is legal. Each child should remain independently checkable.

Start with conservative parallelism. Parallelize only operations with clear independence and bounded resource use. Optimization is valuable when it reduces a measured bottleneck, not when it makes state transitions harder to reason about for no demonstrated benefit.

Predict

Two tool calls both write to the same account balance. What is the safest default?

Serialize or otherwise coordinate the conflicting effects instead of running them independently in parallelRun both immediately because each action was individually authorizedLet the model decide after both writes which result should count

Run the local Lab

Run:

python labs/notebooks/level-12/l12-07-parallel-tools.py

The Lab models four operations and a dependency graph. Run it to see two independent reads grouped into one wave. Then add a dependency from inventory_lookup to user_profile and rerun. Observe that the scheduler creates an additional wave rather than violating the new dependency.

Loading lab…

Write the core logic yourself

Open:

labs/notebooks/level-12/l12-07-parallel-tools-exercise.py

Implement dependency-wave scheduling so independent work can run together, dependent work waits, and a cycle cannot silently stall the scheduler.

Run:

python3 labs/notebooks/level-12/l12-07-parallel-tools-exercise.py

The starter intentionally fails at its TODO boundary. A completed implementation ends with a PASS: marker. Compare with the solved deterministic Lab only after attempting the implementation yourself.

Quick Check

1. When are two tool calls good candidates for parallel execution?

Whenever both tool names appear in the same model responseOnly when both actions are side effects on the same recordWhen inputs and effects are independent

2. What should a join policy define?

How to make all parallel branches share one operation IDWhich results are required and how failures are handledHow to leave partial failures out of the trace

3. Why trace parallel calls as child spans?

To remove the need for task identifiersTo link each call to its parent while showing overlapTo force calls to run sequentially

0 of 3 questions answered.

Explain it back

Give an example with three tool calls: two that can run together and one that must wait. Explain the dependency and what the harness should do if one parallel read fails.

Key Takeaways

• Parallelize only work with clear independence.
• Bound concurrency, cost, and time across the whole batch.
• Conflicting side effects need serialization or coordination.
• Join and cancellation rules should be explicit.
• Every child action keeps its own permission and tracing checks.

Next Lesson

Next, L12.8 — Delegation and Subagents turns one large goal into bounded subtasks without giving away uncontrolled authority.

References

• OpenTelemetry, Specifications.