How to Build with LLM APIs in 2026: A Developer’s Guide

Building a feature on top of a large language model means calling an LLM API — sending messages, handling streaming responses, wiring up tools, and controlling cost. The mechanics are similar across providers. This guide covers the core concepts with code, using Anthropic’s Messages API as the running example, plus the state of the 2026 model landscape. Here are 14 things to know to build with LLM APIs.

1. Know the model landscape (and check the pricing page)

In 2026 the flagship families are Anthropic’s Claude (Opus 5, Sonnet 5, Haiku 4.5, and the top-tier Fable 5), OpenAI’s GPT-5 series (including the Codex-tuned coding models), and Google’s Gemini 3 series. They’re reasoning models with ~1M-token context. Exact IDs, context sizes and prices move — verify them on the provider’s docs/pricing page before you hard-code anything.

// Anthropic model IDs (2026):
//   claude-opus-5    — flagship, complex/agentic work
//   claude-sonnet-5  — balanced, most everyday coding
//   claude-haiku-4-5 — fast + cheap
// Confirm current IDs/prices at the provider docs.

2. Understand the messages shape

A request is a list of role-tagged messages (system, user, assistant) plus the model and an output cap. The API is stateless — you resend the conversation history each turn, so you own memory and context management, not the API.

from anthropic import Anthropic
client = Anthropic()  # reads ANTHROPIC_API_KEY

resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    system="You are a concise assistant.",
    messages=[{"role": "user", "content": "Explain embeddings in one sentence."}],
)
print(resp.content[0].text)

3. Think in tokens

Billing and context limits are counted in tokens (~three-quarters of a word in English), not characters. Count with the provider’s own tokenizer/endpoint — don’t assume one provider’s counter matches another’s. Tokens are also how you budget cost and stay under the context window.

# count before sending, to budget cost and context
count = client.messages.count_tokens(
    model="claude-sonnet-5",
    messages=[{"role":"user","content": long_text}],
)
print(count.input_tokens)

4. Stream responses

Stream tokens as they generate (Server-Sent Events) to render output progressively and to avoid HTTP timeouts on long generations. Streaming is essential for chat UIs and any request with a large max_tokens.

with client.messages.stream(
    model="claude-sonnet-5", max_tokens=2048,
    messages=[{"role":"user","content":"Write a haiku about APIs."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="")

5. Set max_tokens and control length

max_tokens hard-caps the output length (and on reasoning models, thinking plus output together). Set it deliberately: too low truncates the answer, too high risks runaway cost and latency. It’s your primary guardrail on a single call’s size.

resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=512,   # cap the answer length/cost
    messages=[...],
)

6. Use tool calling to connect real systems

Declare typed tools and the model emits a structured call; your code runs it and returns the result in the next turn. This is how you give a model live data and the ability to act — the foundation of every agent. Return errors as results so the model can recover.

tools = [{"name":"get_order","description":"Look up an order by id",
  "input_schema":{"type":"object","properties":{"id":{"type":"string"}},"required":["id"]}}]
resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024,
  tools=tools, messages=[{"role":"user","content":"Where is order A123?"}])
# if resp.stop_reason == "tool_use": run it, send the result back

7. Constrain output with structured/JSON schemas

For anything a program consumes, constrain the response to a JSON Schema so you get guaranteed-parseable output — no regex, no retries on malformed JSON. Every major provider supports this now (Anthropic output formats, OpenAI strict:true, Gemini responseSchema).

// pass a schema; the API guarantees conforming keys
output_config = {
  "format": {"type": "json_schema", "schema": INVOICE_SCHEMA}
}

8. Add embeddings for search and RAG

Embeddings turn text into vectors for semantic search and retrieval-augmented generation. Note the provider split: OpenAI (text-embedding-3-*) and Google (gemini-embedding-*) offer embeddings, while Anthropic recommends third parties (for example Voyage AI). See our RAG guide.

from openai import OpenAI
oai = OpenAI()
vec = oai.embeddings.create(
    model="text-embedding-3-large", input="How do refunds work?"
).data[0].embedding   # store in a vector DB

9. Cache stable prompt prefixes

Prompt caching stores a stable prefix (a big system prompt, a document, tool definitions) so repeated calls bill at a fraction of the price and return faster. Keep the cached prefix byte-stable — any change invalidates it — and put the variable part last.

# mark the last stable block as cacheable (Anthropic)
system = [{"type":"text","text": big_context,
          "cache_control": {"type":"ephemeral"}}]
# repeat calls reuse it at ~a fraction of input cost

10. Control cost deliberately

The main levers are durable even as prices change: use a smaller model tier where it suffices, cache stable prefixes, batch non-urgent work (async batch APIs are typically much cheaper), and cap max_tokens/effort. Measure cost per request and set budgets — LLM spend scales with usage fast.

# batch async jobs for a large discount vs real-time calls
batch = client.messages.batches.create(requests=[...])
# + prompt caching + right-sized model = the big savings

11. Handle rate limits and retries

APIs enforce per-org requests-per-minute and tokens-per-minute limits. Retry 429 and 5xx with exponential backoff (the SDKs do this by default), respect the retry-after header, and never retry 4xx client errors — fix the request instead.

try:
    resp = client.messages.create(...)
except RateLimitError as e:
    # back off using retry-after; SDK retries automatically too
    wait = int(e.response.headers.get("retry-after", 2))

12. Check stop reasons and refusals

Always inspect why generation stopped before reading content. A response can hit the token cap (max_tokens), finish a tool call (tool_use), or be declined (refusal). Handling the stop reason prevents you from treating a truncated or refused response as a complete answer.

if resp.stop_reason == "refusal":
    handle_declined()
elif resp.stop_reason == "max_tokens":
    continue_generation()   # you were truncated

13. Control reasoning depth with an effort budget

The 2026 knob for the quality-vs-cost/latency tradeoff is the thinking/effort budget: thinking + effort (Anthropic), reasoning.effort (OpenAI), thinkingLevel (Gemini). Turn it up for hard problems, down for simple, latency-sensitive calls — it often matters more than which model tier you pick.

resp = client.messages.create(
    model="claude-opus-5", max_tokens=4096,
    output_config={"effort": "high"},   # deeper reasoning
    messages=[...],
)

14. Standardize tool/data access with MCP

The Model Context Protocol (MCP) — introduced by Anthropic in late 2024 and now a vendor-neutral standard under the Linux Foundation, supported across major providers — is a common client/server way to connect models to tools and data. Instead of custom glue per model x tool, any MCP client can talk to any MCP server (GitHub, Postgres, Slack, your internal APIs).

// Point an MCP client at a server and its tools appear to the model.
// Pre-built servers exist for GitHub, Postgres, Slack, Drive, etc.
// "USB-C for AI tools" — one protocol, many integrations.

Building with LLM APIs — checklist

• Pick a model tier for the job; confirm current IDs/prices on the provider docs.
• Resend history each turn (the API is stateless); think in tokens.
• Stream long responses; set max_tokens; check the stop reason.
• Use tool calling for live data/actions; constrain output with JSON schemas.
• Add embeddings for search/RAG; cache stable prefixes to cut cost.
• Handle rate limits with backoff; never retry 4xx.
• Tune the reasoning/effort budget; standardize tool access with MCP.

Build and test LLM-powered features in the browser with XCODX Studio — npm packages and 70+ languages, no setup. Then explore how to build AI agents, the RAG guide, and prompt engineering for developers.