How OpenRouter Model Routing Works for AI Agents

OpenRouter gives you one API key and one endpoint for hundreds of language models. Its pricing page lists 500+ models from 80+ providers on paid plans. Behind that endpoint sit three routing layers that decide which provider and which model serve each request. An agent makes hundreds of calls a day. Those layers decide its uptime and price, and the chosen provider also sets its speed. Below is how each layer works and what it costs. The last section covers what routing leaves to you: conversation state.

What is OpenRouter model routing, in one sentence?

OpenRouter model routing is the set of rules that picks a provider endpoint, and sometimes a different model, for each API request based on your preferences and live provider data.

The defaults are documented in OpenRouter's provider routing guide. Without any settings, OpenRouter load balances across the top providers for the model you named. You can override that per request. A provider object changes who serves the model. A list of fallback models or the openrouter/auto slug changes the model itself.

How do provider routing, model fallbacks and the Auto Router differ?

They act at different levels. Provider routing keeps your model and changes who serves it. Fallbacks and the Auto Router change the model.

Sources: OpenRouter's guides to provider routing and model fallbacks. The Auto Router column follows its own docs, and billing follows the pricing page.

How does OpenRouter choose a provider?

By default it optimizes for price while avoiding recent outages. The docs describe three steps. OpenRouter first prefers providers with no significant outage in the last 30 seconds. Among those, it picks from the cheapest candidates, weighted by the inverse square of the price. The rest become fallbacks.

The inverse square makes cheap providers win most of the time. OpenRouter's own example has providers at $1 and $3 per million tokens. The $1 provider is 9 times more likely to be tried first.

You can turn that off. Setting sort to price, throughput or latency disables load balancing and tries providers in that order. Two model-slug shortcuts do the same job. Adding :nitro to a model slug sorts by throughput and also makes priority endpoints eligible. The :floor suffix sorts by price and opens up cheaper flex endpoints. Other provider fields filter the pool. They can exclude providers that may store data, limit routing to zero data retention endpoints, or set a maximum price.

Tool calls get special handling. When a request includes tools, OpenRouter makes a best effort to pick providers known to support tool use. The same model can handle tool calls differently from one host to the next, so agents benefit most.

How do model fallbacks work?

You pass a models array in priority order. If the first model returns an error, OpenRouter tries the next one. The fallback docs say any error can trigger a fallback. Their examples include context length errors and moderation flags. Rate limits and downtime count too. If the last model also fails, you get its error.

Billing follows the model that answered. The pricing FAQ says an attempt that errors is not charged for its model tokens. If a free model is followed by a paid one and OpenRouter falls back, you pay the paid model's price.

Fallbacks swap the model mid-task without asking. For an agent, the next turn may run on a model with a different tool-calling style or a smaller context window. Pick fallbacks that can actually do the job, and log which model served each step.

What is the OpenRouter Auto Router?

The Auto Router picks a model for you. A fast classifier assigns each prompt one of about 30 task types, such as code debugging or multi-step planning. OpenRouter then ranks models by what its users actually spend on for that task type over a trailing 7-day window. Your cost tier narrows the band, and the top models become the primary pick plus fallbacks.

The ranking measures spend, so it tracks popularity. When developers move a workload to a new model, the router follows within days. The docs say prompts are classified in flight without retention. You can see the task label by opting in to router metadata with a request header.

What does OpenRouter cost?

OpenRouter says it does not mark up inference. Each model is billed at the provider's list price. The fee comes when you buy credits: 5.5% on the Standard plan and 8% on Business, per the pricing page. The free tier offers 25+ free models with a limit of 50 requests a day.

Bring your own provider keys and the first $25,000 of list-price inference each month carries no fee on Standard and Business. After that the fee is 5%. The Auto Router itself adds no fee on top of the chosen model's rate.

What should you read in the fine print?

Most numbers here come from OpenRouter's own pages. We found no independent study of its routing quality or of how often fallbacks fire in practice.

First, routing interacts with prompt caching. The caching guide says OpenRouter keeps sending a cached conversation to the same provider. That sticky session expires after 10 minutes of inactivity. A fallback to another provider or model starts cold, so a long agent context is billed at the regular input price again.

Second, OpenRouter keeps no conversation state. Its Responses API docs say the API is stateless and you must send the full history with each request. Requests that ask OpenRouter to store a response are rejected.

The bottom line

Use provider routing defaults for most traffic. Set sort or :nitro only where latency matters. Add a short fallback list of models you have tested on your own tasks. Treat the Auto Router as a prototype tool and write an explicit routing policy for production. Before you size the bill, count the credit fee plus the cold-cache cost of fallbacks.

How do you keep agent memory when the model keeps changing?

OpenRouter makes switching models easy, and its stateless API leaves conversation state to you. Each request carries the full history, and a fallback can hand the next turn to a model that has never seen your project. Anything the agent learned last week exists only if your application stored it and sends it again. Packing that history into every prompt gets billed in full after each cache miss.

memU keeps that knowledge in a layer that belongs to the agent host, so it stays put when the model behind the host changes. A scheduled background task slices new session logs into jobs. The agent distills them into memory and skill Markdown, which you can read and edit. memU makes no LLM calls itself. It stores and embeds what the agent prepared, then retrieves it when asked.

A standing instruction in the host's instruction file, such as AGENTS.md, tells the agent to retrieve before answering. Supported hosts include Hermes Agent and OpenClaw. Our post on why memU leaves the thinking to the agent covers the design.

The hosted version is free and syncs across devices. Get an API key at memu.so and send your agent one message:

Read https://memu.pro/SKILL.md, follow its instructions to install and configure memU, API Key is YOUR_MEMU_API_KEY.

The self-hosted path and the host support matrix are in the memU GitHub repository.

FAQ

Does OpenRouter charge more than the provider?

No markup on inference, according to OpenRouter. You pay the provider's list price. A fee applies when you buy credits: 5.5% on Standard, 8% on Business.

Do failed fallback attempts cost money on OpenRouter?

No. OpenRouter says an attempt that errors is not charged for model tokens, so you pay for the model that answered. Some auxiliary services that already ran, such as web search, may still be billed.

How does the OpenRouter Auto Router pick a model?

It classifies the prompt into one of about 30 task types. It then picks the models OpenRouter users spent the most on for that task over the last 7 days, within your cost tier.

Does OpenRouter remember my conversation?

No. OpenRouter's Responses API is stateless, so every request must include the full conversation history. Your application or agent host has to keep the memory.

How many models does OpenRouter support?

The pricing page lists 500+ models from 80+ providers on paid plans. The free tier offers 25+ free models from 4 providers.