API gateways in the agent era — what actually changed, and what's just a checkbox

The MCP specification's largest revision since launch is dated 28 July 2026 — this coming Tuesday. Gartner expects three quarters of API gateway vendors to ship MCP features by the end of the year. Every gateway product page has grown an "AI" tab in the last eighteen months.

Very little of this is about gateways getting better. Most of it is about a category racing to attach itself to a traffic pattern it doesn't fully understand yet.

It's worth separating the parts that are genuinely new from the parts that are a feature list assembled by competitors reading each other's changelogs.

The inversion

For twenty years a gateway's job was defensive. Clients were untrusted, backends were precious, and everything in between — auth, rate limits, quotas, WAF rules — existed to keep the second safe from the first. The client was assumed to be a program written by a human who had read the docs beforehand, signed up, agreed to terms, and hard-coded the endpoints they needed.

Every one of those assumptions is now optional.

An agent reads the documentation at runtime. It decides which endpoint to call while the request is in flight. It has no prior relationship with the API and frequently no human in the loop at the moment of the call. It will retry aggressively, fan out in parallel, and abandon a request halfway through because the model changed its mind.

So the gateway's job inverts. It is no longer only protecting the backend from the client. It now has to make the backend legible to a client that arrives without context, and settleable with a client that has no account.

That reframing sorts the feature list quickly. Some of the popular capabilities serve the inversion. Others are optimisations for a traffic shape that already existed.

From the platform engineer's seat: the shape of the traffic

The first thing that breaks is not cost or security. It's timing.

Traditional gateways are built around a request that completes in milliseconds and produces one response. Agent traffic produces long-lived streams, and a lot of them. A streamed completion holds a connection open for thirty seconds or more, which means connection-pool maths that worked for a decade quietly stops working. Idle timeouts that were generous for a REST call are aggressive for a stream. Buffering proxies — the default in more places than people expect — turn a streaming response into a single delayed blob, and nothing errors. It just feels broken.

Then there's the retry behaviour. Agents retry more than humans do, and they retry differently: not "the network failed" but "the answer wasn't good enough." That traffic looks identical to a client bug from the gateway's side, and the standard defence — aggressive rate limiting — produces a failure loop where the agent retries harder because it's being throttled.

The MCP-behind-a-gateway story is worse, and it's the honest gap in the current spec. Enterprises putting MCP servers behind gateways, security proxies or load balancers have hit undefined behaviour around authorization propagation, session affinity and inspection boundaries. Those aren't exotic edge cases. They're what happens the first time you put a normal production topology in front of an MCP server. The July spec moves the auth story much closer to standard OAuth and OpenID Connect deployments, which is the right direction, but a specification landing this week is not the same as a fleet of gateways implementing it correctly by autumn.

If you're evaluating a gateway right now, the questions that separate real support from a checkbox are boring ones. Does it stream without buffering, by default? What happens to a session when a connection is re-established through a different node? Does it propagate the caller's identity to the upstream, or does it terminate auth and forward its own?

From the security team's seat: the identity problem

The security conversation about AI gateways is dominated by prompt injection. It's a real threat, and gateway-level filtering is a reasonable layer. It is also the easier problem, and it gets attention partly because it's tractable enough to demo.

The harder one is delegation.

When an agent calls an API on a user's behalf, the gateway sees one credential. Behind that credential is a chain: a human authorised an agent, the agent was orchestrated by a framework, the framework called a tool, and the tool called your API. Traditional API keys flatten that entire chain into a single opaque string. Every audit question that matters afterwards — who authorised this, on whose behalf, under what limit, and was that limit still valid at execution time — is unanswerable from the key alone.

This is why the interesting movement in the payments protocols is not the payment part. Google's AP2 works through cryptographically signed mandates — a machine-readable statement of what an agent was permitted to do, signed by the party that permitted it. That's a delegation primitive that happens to settle money. The Machine Payments Protocol that Stripe and Tempo launched in March uses a sessions model where an agent pre-authorises a spending ceiling and streams micropayments underneath it — again, a bounded grant rather than a bearer token with unlimited scope.

Read those as authorization designs and they're more significant than read as payment rails. A key that says "this agent may spend up to $40 on this task, until Friday" is a fundamentally better security object than a key that says "this is a valid customer."

Most gateways today can't express that. Scoped keys and per-key rate limits get part of the way — restricting which APIs a key can reach, and how fast — but the ceiling is usually a rate, not a budget, and it rarely carries a purpose or an expiry tied to a task.

From the finance seat: quotas are the wrong unit

Token-based rate limiting is on every AI gateway's feature list, and it deserves to be. Counting requests when the cost of a request varies by three orders of magnitude is not measurement, it's superstition.

But the framing is usually too narrow. The real problem is not "limit tokens per minute." It's that the unit of value stopped matching the unit of billing across the entire stack.

An agent conversation triggers hundreds of micro-activities with sub-cent costs. A research agent hitting a market data endpoint two hundred times a day does not fit a monthly tier designed for a developer making twenty calls a week. That mismatch is why Amazon, X and Reddit all restructured API pricing in early 2026 toward usage-based, tiered systems, and why credit-based pricing has become the default shape for AI-native products.

Which brings up semantic caching — the most oversold item on the list.

The pitch is good: match an incoming prompt against previous ones by meaning rather than exact text, and serve the cached response. On genuinely repetitive workloads, the savings are real and large.

The problem is that "means the same thing" is a judgement call being made by an embedding-similarity threshold. Set it loose and you serve a subtly wrong answer with total confidence and no error anywhere in your logs. Set it tight and the hit rate collapses to roughly what an exact-match cache would have given you. Somewhere in the middle is a threshold that works for your workload, and finding it is empirical work that no vendor can do for you.

Semantic caching on a support-macro workload: obviously correct. On anything where the answer depends on freshness, on the user, or on a small detail in a long prompt: at minimum, instrument the hit rate and sample the hits for correctness before you trust the savings number on the dashboard.

Multi-provider routing carries a quieter version of the same problem. Failing over from one model to another when a provider rate-limits you is excellent for availability and terrible for consistency — prompts that were tuned against one model behave differently on another, and the fallback path is exactly the one nobody tested. Failover is a reliability feature that quietly becomes a correctness feature. Treat the fallback as a distinct configuration you evaluate, not a checkbox you enable.

From the API provider's seat: the discovery problem

Everything above is written from the perspective of someone consuming APIs. Flip it around, because this is the half that gateway vendors mostly aren't addressing.

If you publish an API in 2026, your prospective customer is increasingly not a developer browsing your docs. It's a model, deciding at runtime whether your endpoint can answer the question in front of it. That changes what "good documentation" means.

A human developer will tolerate documentation that requires reading three pages and inferring the rest. An agent needs a machine-readable contract it can fetch, parse and act on in one step. In practice that means a stable OpenAPI document at a predictable URL — not a spec buried behind a docs portal's JavaScript, not a Postman collection you have to request access to, not an SDK. A fetchable spec, and an honest one, where the response examples reflect what the endpoint actually returns.

The gap between "we have OpenAPI" and "an agent can use our OpenAPI" is mostly honesty about responses. A spec that documents parameters but leaves the response shape undefined tells an agent nothing about whether the call is worth making. We ran into exactly this on our own catalog: the parameters were well documented, the 200 example wasn't, and a tutorial we wrote against it had to hedge on field names until the provider filled the gap. Once the example existed, the same spec became genuinely actionable — and it surfaced a naming inconsistency between the request and response that had been invisible to everyone until something had to read both.

That's the shape of the discovery problem. It isn't a marketing problem. It's a contract-completeness problem, and it's the cheapest high-leverage work available to most API providers right now.

The part still mostly unsolved: settlement

Say an agent has found your API and understood the contract. It still cannot use it.

Between "this endpoint can answer my question" and "I have called it" sits an account, an email verification, a credit card form, a plan selection and a set of terms — a flow designed end to end for a human with a browser. Every one of those steps assumes a person is present.

This is the actual promise of the payment protocols, and why there are suddenly four of them. x402, which Coinbase shipped in May 2025, uses HTTP's long-dormant 402 status to make a request itself payable: call an endpoint, get a 402 with a price, pay, retry. Coinbase reported roughly $50 million in cumulative volume by late April 2026, Stripe integrated it on Base in February, and Cloudflare supports it. OpenAI and Stripe's ACP standardises agent-to-merchant checkout. AP2 and MPP cover the authorization and streaming-micropayment halves respectively.

They are not really competitors — an agent might plausibly use AP2 to prove it was authorised, ACP to check out, and x402 or MPP to actually move value. What they share is the assumption that discovery, authorization and settlement all have to work without a human at the keyboard.

Where this lands for gateway operators is a design question worth thinking about before it becomes urgent: what is the smallest unit you can sell, and how quickly can a new caller start paying for it? If the answer involves a signup form, you are not addressable by this traffic, no matter how good your rate limiting is.

There's a middle position that doesn't require adopting any of the four protocols, which is where we've landed on our own platform: a single prepaid balance that works across every API a customer subscribes to, metering in whatever unit the API actually sells — results, pages, posts, tokens — and reporting consumption on the response itself, so the caller knows what a request cost without waiting for an invoice. One key, one balance, many providers. It doesn't solve autonomous onboarding. It does remove the per-provider account, contract and invoice, which is the part that makes multi-API consumption painful today, and it means metering is a first-class concern rather than something reconciled monthly.

The per-API OpenAPI documents sit at predictable URLs for the same reason: whatever eventually wins on payments, the fetchable-contract part is required either way.

What we'd actually check

If you're evaluating a gateway this quarter, the list that separates products is shorter than the marketing suggests:

• Streaming without buffering, by default. Verify it, don't read it.
• Identity propagation. Does the upstream learn who the caller was, or only that the gateway approved them?
• Budgets, not just rates. Can a credential carry a spending ceiling and an expiry, or only requests-per-second?
• Metering in the unit you sell. If you sell results and meter requests, your margins depend on a coincidence.
• MCP behaviour under a real topology. Sessions, reconnects, auth propagation — not "MCP: yes" on a comparison table.
• Semantic caching you can measure. Hit rate and a sampling mechanism for correctness, or don't turn it on.
• A contract an agent can fetch. Stable spec URL, real response examples.

The first four are old gateway concerns, sharpened. The last three are genuinely new. Nearly every vendor in the category will tell you they do all seven; the ones worth paying for can show you the third and the sixth working.

Where this is heading

The uncomfortable read on all of this is that "AI gateway" may not survive as a category. Token counting, semantic caching and provider failover are features, and features get absorbed. The deployment landscape already splits three ways — managed cloud, self-hosted open source, and enterprise platform extensions — which is usually what a category looks like shortly before its distinctive features become table stakes everywhere.

What won't get absorbed is the harder shift underneath: an API's addressable market now includes software that has never heard of you, arrives with no account, reads your contract in one fetch, and decides in a few hundred milliseconds whether you're worth calling. Rate limiting doesn't help you there. A complete, honest, machine-readable contract does — and so does being able to take payment from something that will never fill in a form.

Most of the industry is currently building the first half. The second is where the interesting problems are.