Web and API Tool Patterns

Goal

Turn a remote web or API action into a small, predictable tool: fixed operations, hidden credentials, normalized results, bounded waiting, and explicit handling for pages and failures.

Start with one remote job

Suppose an assistant only needs to look up the weather for a city. Giving the model a tool called fetch_any_url(url) is much broader than the job requires. The model could choose the wrong endpoint, follow an unexpected link, or send data somewhere the application never intended.

A narrower tool could look like:

get_weather(city="Helsinki")
→ {temperature_c: 8, condition: "rain"}

The application—not the model—chooses the weather provider, adds authentication, sets the timeout, follows the provider's pagination rules if needed, and converts the provider's response into the small result the workflow actually uses.

This is the main pattern for web and API tools: wrap a large remote API in a task-shaped operation. The model chooses among allowed jobs and supplies the job's arguments. Application code owns network destinations, credentials, retries, response limits, and normalization.

That separation makes failures easier to understand too. If get_weather times out, the workflow sees a weather-tool timeout. It does not have to reason about an arbitrary URL, a provider-specific error body, and hidden authentication details all at once.

Keep credentials outside model-visible arguments

An API token should come from secret/application configuration, not from a model-produced field.

The tool call might contain:

{"order_id":"4172"}

while trusted application code adds authentication when it performs the remote request. The model never needs to see the credential.

Normalize status and error behavior

Remote services return many status codes and payload shapes. Convert them into a small stable result model such as:

{"ok":true,"data":{"order_id":"4172","status":"shipped"}}

or:

{"ok":false,"category":"rate_limit","retryable":true}

The workflow should not depend on an accidental error-page string.

Pagination is part of correctness

A search endpoint may return only the first page. If the user asks for “all open incidents,” reading one page and reporting that as complete is a logic error.

Record whether results are complete, truncated, or paginated. Put a cap on how much the tool can fetch in one workflow step.

Timeouts and freshness belong in the trace

Network calls need explicit timeouts. Results that are time-sensitive should carry an observation time or source version when available.

A result from ten minutes ago may be sufficient for a product description and unacceptable for a rapidly changing operational status. The task determines the freshness requirement.

Separate remote identity from model-visible data

A model often needs to choose what to fetch without controlling where credentials or tenant identity come from. For example, get_order("4172") can use the authenticated account and fixed service base URL held by the application. Exposing tenant_id, bearer token, and arbitrary host as model arguments creates flexibility the task does not need.

This separation also improves replay. A recorded trace can store the logical tool call and a sanitized result without copying secrets into evaluation fixtures. When a live integration changes providers, the model-facing interface can remain stable even though authentication and transport code change underneath.

Remote data needs size and completeness policy

APIs can return unexpectedly large collections or nested payloads. Define item, byte, and page limits before adding results to context. If the limit is reached, tell the workflow that the result is partial rather than silently pretending it is complete. A later step can request a narrower query or another page when the task justifies it.

Predict

A support assistant only needs to read orders from one service. Which interface is easier to secure?

A narrow get_order(order_id) wrapper with fixed service configurationA generic request(url, method, headers, body) toolA shell command tool with network access

Run the local Lab

python labs/notebooks/level-10/l10-07-api-patterns.py

The Lab pages through three recorded items, inc-1 to inc-3, with page sizes 2 and 1.

1. Run the command. With page_size 2, the first page is ['inc-1', 'inc-2'] with complete: False. With page_size 1, it is ['inc-1'], also complete: False. Neither first page is the whole result.
2. Ask for the second page instead: change first = fetch_page(0, page_size) to first = fetch_page(1, page_size) and rerun.
3. With page_size 2, page 1 is ['inc-3'] and complete: True. With page_size 1, page 1 is ['inc-2'] and still complete: False.
4. Explain what goes wrong if a tool reads only the first page and reports “these are all the incidents.” Then connect it to L10.5: a timeout while fetching a page may be retried, but a permanent not_found should not be. Change the line back afterward.

Loading lab…

Quick Check

1. Where should an API credential normally come from?

An external webpage or tool result containing a credential-like stringA model-produced argument copied from the conversation whenever the user supplies a tokenTrusted application configuration or secret storage

2. Why normalize remote response shapes?

To recover from every timeout without a separate retry or completion policyTo make remote JSON easier to read in logsTo give the workflow a stable success/error interface

3. What is a risk of ignoring pagination?

The model's tokenizer changesThe workflow can mistake a partial result set for a complete oneAuthentication stops being checked for later pages

0 of 3 questions answered.

Key Takeaways

• Prefer task-shaped API wrappers over open-ended network tools.
• Keep credentials and fixed service configuration outside model arguments.
• Normalize remote successes and failures into stable shapes.
• Pagination and result completeness are correctness concerns.
• Record timeout and freshness information when the task depends on it.

Next Lesson

Next, treat code execution as an even more powerful tool and reduce its filesystem, process, network, time, and output authority.

References

• OpenAPI Initiative, OpenAPI Specification 3.1.0.
• IETF, HTTP Semantics — RFC 9110.