MCP in production: gateways, limits and audit
An allowlist, per-user rate limits, header checks and an audit log in front of many servers.
What it covers6
Why it matters
One MCP server on one laptop needs no gateway. A company with twenty servers and a thousand users does. It wants to decide, in one place, which tools each team may call, how often, with whose token, and to keep a record of every call. A gateway is that one place: a service in front of the servers that checks each request before it goes through and writes down what happened. The mirrored headers from the HTTP lesson exist so a gateway can do its job without reading the body.
What you'll be able to do
- Say what a gateway in front of MCP servers checks, and why the mirrored headers let it do that cheaply
- Write a gateway that checks the token's audience, an allowlist, a per-user per-tool rate limit and the headers, and logs an audit line
- Spread requests over stateless server copies, and read usage, errors and slow calls from the audit log
- Follow one request across host, gateway and server with a trace id
Show the answer
In one gateway in front of the servers
One place that sees every request can apply one policy and write one audit log. Copying it into twenty servers means twenty versions to keep in step.
The idea: a gateway reads the envelope, not the letter
This rests on Transports II: Streamable HTTP: every request names its method and its tool in headers (Mcp-Method and Mcp-Name), and the server refuses a request whose headers disagree with its body. Those headers are for the gateway. It can decide a lot from the envelope without parsing a single JSON body:
- Who is asking, and for which server? The token's audience must be the gateway. The gateway then calls the servers behind it with its own token, never the user's, which is the rule against passthrough from Securing remote servers with OAuth.
- Is this tool allowed? An allowlist per team, read from
Mcp-Name. - Is this user calling it too often? A rate limit per user and per tool.
- Which copy answers? Modern MCP has no sessions, so any copy of a server can answer any request. The gateway spreads requests over the copies.
- What happened? One audit line per request: who, which tool, the status, how long, which copy.
The header check is also a security check. If the gateway allowed get_campaign by its header and the server ran delete_campaign from the body, the allowlist would mean nothing. So the gateway rejects a mismatch itself, with a 400, before anything is forwarded.
limit, window = 3, 60 hits = [0, 10, 20] room = [] for now in (30, 61): recent = [t for t in hits if now - t < window] room.append(len(recent) < limit) print(*room)
Show the answer
False True
At time 30 the three hits are all still inside the 60-second window, so the limit is reached and the request is refused. At time 61 the hit at 0 has aged out, leaving two, so there is room again.
One request through a gateway to two server copies, then the same request with a header that disagrees with its body. Try the edit.
What a gateway does not do
A gateway is a checkpoint, not a cure. It does not read the model's mind: it can't tell that a tool result carries an injected order, which is a problem for the host (Lab: attack and defend a host). It does not make a slow tool fast. It does not replace a server's own checks, because the gateway can be bypassed by anything that reaches a server directly, so the servers should accept only the gateway's token. And it adds a hop: every request now pays for one more network trip and one more thing that can fail, so the gateway itself must be stateless, fast and run in more than one copy.
What it does give you is a single place to change a rule. When a team's allowlist changes, a limit is lowered or a token's audience moves, you change one service instead of twenty.
Show the answer
Nothing, if a caller can reach the server directly
A checkpoint only checks what goes through it. The servers should accept only the gateway's own token, so the gateway is the only road in.
Write a sentence or two, then compare it with a good answer in the lesson.
In Python: a gateway, an audit log and a chart
The setup holds a small gateway: tokens with audiences (as in Securing remote servers with OAuth), stateless server copies, and a Gateway class whose decide runs the checks in a fixed order: token, headers, allowlist, rate limit. Open it above the first cell. The order matters: a request with a wrong token must never reach the allowlist, and a request for a tool that isn't allowed must never use up a rate-limit slot.
The first cell sends one request of each kind and prints what the gateway answered. call builds a request with the mirrored headers:
Setup (runs before every cell) · 92 lines
The lesson runs this once, before its first cell, and every cell, drill and example starts with what it defines. A cell that changes a setup object changes it for the cells after it, until the page reloads.
import base64 import hashlib import hmac import json KEY = b"practice-auth-key" GATEWAY, UPSTREAM = "https://gateway.shop.example/mcp", "https://shop.internal/mcp" LATENCY_MS = {"get_campaign": 40, "search_docs": 120, "export_report": 900} def issue(aud, user, exp=2000): claims = {"aud": aud, "sub": user, "exp": exp} body = base64.urlsafe_b64encode(json.dumps(claims).encode()).decode() return body + "." + hmac.new(KEY, body.encode(), hashlib.sha256).hexdigest() def claims_of(token): body, _, sig = token.partition(".") good = hmac.new(KEY, body.encode(), hashlib.sha256).hexdigest() if not hmac.compare_digest(sig, good): return None return json.loads(base64.urlsafe_b64decode(body)) def make_copy(number): """One of several identical, stateless server copies behind the gateway.""" def handle(token, msg): claims = claims_of(token) if claims is None or claims["aud"] != UPSTREAM: return 401, {"error": {"code": -32001, "message": "bad upstream token"}} args = msg["params"].get("arguments", {}) failed = bool(args.get("fail")) text = "export failed" if failed else f"ok from copy {number}" return 200, {"result": {"content": [{"type": "text", "text": text}], "isError": failed}} return handle class Gateway: def __init__(self, copies, allow, limit, window=60): self.copies, self.allow = copies, allow self.limit, self.window = limit, window self.hits, self.audit, self.turn = {}, [], 0 def handle(self, request, now): headers, msg, user = request["headers"], request["body"], request["user"] method, params = msg["method"], msg.get("params", {}) name = params.get("name") status, body, copy = self.decide(headers, msg, user, now) if status is None: copy = self.turn % len(self.copies) self.turn += 1 status, body = self.copies[copy](issue(UPSTREAM, "gateway"), msg) ms = LATENCY_MS.get(name, 5) failed = status != 200 or body.get("result", {}).get("isError", False) self.audit.append({"user": user, "method": method, "tool": name, "status": status, "ms": ms, "copy": copy, "error": failed}) return status, body def decide(self, headers, msg, user, now): """The gateway's own checks. (None, None, None) means: forward it.""" method, name = msg["method"], msg.get("params", {}).get("name") claims = claims_of(request_token(headers)) if claims is None or claims["aud"] != GATEWAY or claims["exp"] <= now: return 401, {"error": {"code": -32001, "message": "bad token"}}, None if headers.get("mcp-method") != method: return 400, {"error": {"code": -32020, "message": "header mismatch"}}, None if name is not None and headers.get("mcp-name") != name: return 400, {"error": {"code": -32020, "message": "header mismatch"}}, None if method != "tools/call": return None, None, None if name not in self.allow: return 403, {"error": {"code": -32001, "message": "tool not allowed"}}, None key = (user, name) recent = [t for t in self.hits.get(key, []) if now - t < self.window] if len(recent) >= self.limit: return 429, {"error": {"code": -32001, "message": "rate limited"}}, None self.hits[key] = recent + [now] return None, None, None def request_token(headers): return headers.get("authorization", "").replace("Bearer ", "") def call(user, tool, token=None, arguments=None, headers_name=None): """A tools/call as the host would send it, with the mirrored headers.""" msg = {"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": tool, "arguments": arguments or {}}} headers = {"authorization": "Bearer " + (token or issue(GATEWAY, user)), "mcp-method": "tools/call", "mcp-name": headers_name or tool} return {"user": user, "headers": headers, "body": msg}
gw = Gateway([make_copy(1), make_copy(2)], limit=3, allow={"get_campaign", "search_docs"}) now = 1000 asked = [("ann", "get_campaign"), ("ann", "get_campaign"), ("bob", "search_docs"), ("ann", "get_campaign"), ("ann", "get_campaign"), ("ann", "delete_campaign")] for i, (user, tool) in enumerate(asked): status, body = gw.handle(call(user, tool), now + i) print(user.ljust(4), tool.ljust(16), status) mismatch = call("ann", "get_campaign", headers_name="search_docs") print("header mismatch:", gw.handle(mismatch, now + 10)[0]) upstream = call("ann", "get_campaign", token=issue(UPSTREAM, "ann")) print("wrong audience: ", gw.handle(upstream, now + 11)[0])
Read the audit log
Every request left an audit line, including the refused ones. This cell sends a day's traffic through the gateway, then counts calls and errors per tool and finds the slowest call, then draws it.
Uses the setup above, 92 lines.
from collections import Counter import matplotlib.pyplot as plt gw = Gateway([make_copy(1), make_copy(2)], limit=100, allow={"get_campaign", "search_docs", "export_report"}) tools = ["get_campaign", "search_docs", "get_campaign", "export_report"] for i in range(16): tool = tools[i % 4] fail = tool == "export_report" and i % 8 == 3 gw.handle(call(["ann", "bob"][i % 2], tool, arguments={"fail": fail}), 1000 + i) gw.handle(call("ann", "delete_campaign"), 1020) calls = Counter(line["tool"] for line in gw.audit) errors = Counter(line["tool"] for line in gw.audit if line["error"]) slowest = max(gw.audit, key=lambda line: line["ms"]) names = sorted(calls) print({name: (calls[name], errors[name]) for name in names}) print("slowest:", slowest["tool"], slowest["ms"], "ms, copy", slowest["copy"]) fig, (left, right) = plt.subplots(1, 2, figsize=(9, 3.2)) left.bar([n for n in names], [calls[n] for n in names], label="calls") left.bar([n for n in names], [errors[n] for n in names], label="errors") left.set_title("Calls and errors by tool") left.set_ylabel("requests") left.legend() right.bar(names, [max(l["ms"] for l in gw.audit if l["tool"] == n) for n in names]) right.set_title("Slowest call by tool") right.set_ylabel("milliseconds") for axis in (left, right): axis.tick_params(axis="x", labelrotation=20) plt.tight_layout() plt.show()
Four numbers answer most production questions: how many calls each tool gets, how many fail, which is slowest and which copy answered. export_report is both the slowest and the one that fails, and delete_campaign was refused before it reached a server, so it shows as an error in the log. Nothing here needed the request bodies.
Show the answer
The gateway refused it with 403: not on the allowlist
The allowlist doesn't include delete_campaign, so the gateway answered 403 itself. The refusal is logged like any other request, which is how an audit finds attempts, not only successes.
copies = 2 served = [turn % copies for turn in range(5)] print(served.count(0))
Show the answer
3
The turns go 0, 1, 0, 1, 0, so copy 0 gets requests 1, 3 and 5. Round robin is even over many requests and off by one over an odd number.
How long until a slot frees up?
Write retry_after(hits, now, window, limit): hits are the times of recent accepted calls. Return 0 if fewer than limit of them are inside the window, otherwise the whole seconds until the oldest hit in the window leaves it.
Uses the setup above, 92 lines.
def retry_after(hits, now, window, limit): return 0 print(retry_after([10, 20, 30], 40, 60, 3))
Which tools are slowest?
Write slowest(audit, k): audit is a list of dicts with tool and ms. Return the names of the k tools with the largest single call, slowest first. Each tool appears once.
Uses the setup above, 92 lines.
def slowest(audit, k): return [] log = [{"tool": "a", "ms": 40}, {"tool": "b", "ms": 900}, {"tool": "a", "ms": 120}] print(slowest(log, 2))
Write a sentence or two, then compare it with a good answer in the lesson.
In production: traces, ownership and what goes wrong
Purpose. Connect the audit line to the rest of the system, and say who owns what.
An audit line tells you a call was slow. A trace tells you why. The 2026-07-28 revision documents OpenTelemetry trace context in _meta: a request may carry traceparent (and tracestate, baggage), so one trace id follows a request from the host through the gateway and the server to the APIs the server calls. The gateway copies the id into its audit line, so the log and the trace join on it.
What a platform team owns. The token settings (audience and scopes), the allowlists per team, the rate limits and the retention of the audit log. Application teams own their tools. When a tool is added, the allowlist change is a reviewed change, and when a limit is hit, the 429 carries a Retry-After so a client waits instead of hammering.
In an engineer's week. Someone asks "who called the refund tool last Tuesday?" The audit log answers in a minute if every request, refused ones included, wrote a line with the user, the tool and the status. What goes wrong is a gateway that logs only successes, a rate limit shared across all users so one script locks out everyone, and a gateway that forwards the user's token because it was easier.
Goes deeper in
- Securing remote servers with OAuth: the audience rule the gateway enforces.
- Testing, designing and shipping servers: testing the gateway like any server.
- Tracing a run end to end: tracing an agent's whole run, with MCP calls as spans.
Show the answer
20
Retry-After is the time until the oldest hit leaves the window: 60 − 40 = 20 seconds. Then there is room for one more call.
Write a sentence or two, then compare it with a good answer in the lesson.
Practice what you just read
- 2 code examples you can edit and run
- a practice exercise that checks your code
- a 5-question quiz
- 9 review cards for spaced repetition
Try a quiz question
Why can a gateway enforce a per-tool allowlist without parsing the request body?
Before this lesson
Where this lesson leads.
575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.
Phase 9 · about 13 hours of study
Model Context Protocol
Build MCP servers, clients and hosts, from the wire format to production
17 lessons 1 free
- JSON-RPC 2.0 in depth FreeRequests, notifications, errors and a dispatcher built from scratch.35 min
- Versions and stateless requestsRevisions by date, the legacy handshake, per-request _meta and version negotiation.30 min
- Discovery, capabilities and the two erasserver/discover, declared capabilities, and talking to legacy and modern software alike.30 min
- Transports I: stdioNewline-delimited JSON over a subprocess's standard streams.30 min
- Transports II: Streamable HTTPOne endpoint, required headers, JSON or event-stream replies.40 min
- Build a server: toolsTool definitions, schemas from type hints, results and two kinds of error.45 min
- Resources and promptsReadable data, URI templates and reusable prompt templates.40 min
- The same server with the official SDKsYour registry and dispatcher on the SDK's decorator shape, in Python and TypeScript.40 min
- Build a clientMetadata on every request, version fallback, paging and caching.40 min
- Hosts: connecting servers to a modelMany servers, one agent: namespacing, routing and approvals.40 min
- Progress, subscriptions and multi-round-trip requestsLong-running work, change notifications and asking the user mid-call.45 min
- Securing remote servers with OAuthProtected resource metadata, PKCE, audiences and scope challenges.45 min
- MCP securityTool poisoning, rug pulls, confused deputies and local-server risks.40 min
- Lab: attack and defend a hostPoisoned tools, injected results, token replay and the defenses that stop each.40 min
- Testing, designing and shipping serversConformance checks, tool design for models, SDKs and deployment.40 min
- MCP in production: gateways, limits and auditAn allowlist, per-user rate limits, header checks and an audit log in front of many servers.35 min
- Capstone: a production MCP serverA complete, tested server for the practice shop.45 min
Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write