Index Ai Academy
Phase 9 · Production MCP

MCP in production: gateways, limits and audit

An allowlist, per-user rate limits, header checks and an audit log in front of many servers.

About 35 minutes · API rules checked November 2026 · A free account opens the whole lesson.
Already have an account? Sign in to open this lesson

What it covers6

Why it matters

One MCP server on one laptop needs no gateway. A company with twenty servers and a thousand users does. It wants to decide, in one place, which tools each team may call, how often, with whose token, and to keep a record of every call. A gateway is that one place: a service in front of the servers that checks each request before it goes through and writes down what happened. The mirrored headers from the HTTP lesson exist so a gateway can do its job without reading the body.

What you'll be able to do

  • Say what a gateway in front of MCP servers checks, and why the mirrored headers let it do that cheaply
  • Write a gateway that checks the token's audience, an allowlist, a per-user per-tool rate limit and the headers, and logs an audit line
  • Spread requests over stateless server copies, and read usage, errors and slow calls from the audit log
  • Follow one request across host, gateway and server with a trace id
Quick check
A company wants to limit how often each employee can call each MCP tool, and keep a record of every call. Where should that logic live?
  1. In the model's system prompt
  2. In one gateway in front of the servers
  3. Nowhere: servers can't be limited
  4. In every server, copied
Show the answer

In one gateway in front of the servers

One place that sees every request can apply one policy and write one audit log. Copying it into twenty servers means twenty versions to keep in step.

The idea: a gateway reads the envelope, not the letter

This rests on Transports II: Streamable HTTP: every request names its method and its tool in headers (Mcp-Method and Mcp-Name), and the server refuses a request whose headers disagree with its body. Those headers are for the gateway. It can decide a lot from the envelope without parsing a single JSON body:

  • Who is asking, and for which server? The token's audience must be the gateway. The gateway then calls the servers behind it with its own token, never the user's, which is the rule against passthrough from Securing remote servers with OAuth.
  • Is this tool allowed? An allowlist per team, read from Mcp-Name.
  • Is this user calling it too often? A rate limit per user and per tool.
  • Which copy answers? Modern MCP has no sessions, so any copy of a server can answer any request. The gateway spreads requests over the copies.
  • What happened? One audit line per request: who, which tool, the status, how long, which copy.

The header check is also a security check. If the gateway allowed get_campaign by its header and the server ran delete_campaign from the body, the allowlist would mean nothing. So the gateway rejects a mismatch itself, with a 400, before anything is forwarded.

Quick check
From the lesson on API reliability: a rate limit counts recent hits in a window and refuses when the count reaches the limit. What does this print?
limit, window = 3, 60
hits = [0, 10, 20]
room = []
for now in (30, 61):
    recent = [t for t in hits if now - t < window]
    room.append(len(recent) < limit)
print(*room)
  1. True False
  2. False False
  3. False True
  4. True True
Show the answer

False True

At time 30 the three hits are all still inside the 60-second window, so the limit is reached and the request is refused. At time 61 the hit at 0 has aged out, leaving two, so there is room again.

Try it now

One request through a gateway to two server copies, then the same request with a header that disagrees with its body. Try the edit.

What a gateway does not do

A gateway is a checkpoint, not a cure. It does not read the model's mind: it can't tell that a tool result carries an injected order, which is a problem for the host (Lab: attack and defend a host). It does not make a slow tool fast. It does not replace a server's own checks, because the gateway can be bypassed by anything that reaches a server directly, so the servers should accept only the gateway's token. And it adds a hop: every request now pays for one more network trip and one more thing that can fail, so the gateway itself must be stateless, fast and run in more than one copy.

What it does give you is a single place to change a rule. When a team's allowlist changes, a limit is lowered or a token's audience moves, you change one service instead of twenty.

Quick check
A server behind the gateway also accepts requests from anywhere on the company network, with any valid user token. What does the gateway's allowlist protect?
  1. Only the audit log
  2. Only the model
  3. Nothing, if a caller can reach the server directly
  4. Everything, because the gateway is in front
Show the answer

Nothing, if a caller can reach the server directly

A checkpoint only checks what goes through it. The servers should accept only the gateway's own token, so the gateway is the only road in.

In your own words
Why must the gateway call the servers behind it with its own token and not forward the user's?

Write a sentence or two, then compare it with a good answer in the lesson.

In Python: a gateway, an audit log and a chart

The setup holds a small gateway: tokens with audiences (as in Securing remote servers with OAuth), stateless server copies, and a Gateway class whose decide runs the checks in a fixed order: token, headers, allowlist, rate limit. Open it above the first cell. The order matters: a request with a wrong token must never reach the allowlist, and a request for a tool that isn't allowed must never use up a rate-limit slot.

The first cell sends one request of each kind and prints what the gateway answered. call builds a request with the mirrored headers:

Setup (runs before every cell) · 92 lines

The lesson runs this once, before its first cell, and every cell, drill and example starts with what it defines. A cell that changes a setup object changes it for the cells after it, until the page reloads.

import base64
import hashlib
import hmac
import json

KEY = b"practice-auth-key"
GATEWAY, UPSTREAM = "https://gateway.shop.example/mcp", "https://shop.internal/mcp"
LATENCY_MS = {"get_campaign": 40, "search_docs": 120, "export_report": 900}


def issue(aud, user, exp=2000):
    claims = {"aud": aud, "sub": user, "exp": exp}
    body = base64.urlsafe_b64encode(json.dumps(claims).encode()).decode()
    return body + "." + hmac.new(KEY, body.encode(), hashlib.sha256).hexdigest()


def claims_of(token):
    body, _, sig = token.partition(".")
    good = hmac.new(KEY, body.encode(), hashlib.sha256).hexdigest()
    if not hmac.compare_digest(sig, good):
        return None
    return json.loads(base64.urlsafe_b64decode(body))


def make_copy(number):
    """One of several identical, stateless server copies behind the gateway."""
    def handle(token, msg):
        claims = claims_of(token)
        if claims is None or claims["aud"] != UPSTREAM:
            return 401, {"error": {"code": -32001, "message": "bad upstream token"}}
        args = msg["params"].get("arguments", {})
        failed = bool(args.get("fail"))
        text = "export failed" if failed else f"ok from copy {number}"
        return 200, {"result": {"content": [{"type": "text", "text": text}],
                                "isError": failed}}
    return handle


class Gateway:
    def __init__(self, copies, allow, limit, window=60):
        self.copies, self.allow = copies, allow
        self.limit, self.window = limit, window
        self.hits, self.audit, self.turn = {}, [], 0

    def handle(self, request, now):
        headers, msg, user = request["headers"], request["body"], request["user"]
        method, params = msg["method"], msg.get("params", {})
        name = params.get("name")
        status, body, copy = self.decide(headers, msg, user, now)
        if status is None:
            copy = self.turn % len(self.copies)
            self.turn += 1
            status, body = self.copies[copy](issue(UPSTREAM, "gateway"), msg)
        ms = LATENCY_MS.get(name, 5)
        failed = status != 200 or body.get("result", {}).get("isError", False)
        self.audit.append({"user": user, "method": method, "tool": name,
                           "status": status, "ms": ms, "copy": copy, "error": failed})
        return status, body

    def decide(self, headers, msg, user, now):
        """The gateway's own checks. (None, None, None) means: forward it."""
        method, name = msg["method"], msg.get("params", {}).get("name")
        claims = claims_of(request_token(headers))
        if claims is None or claims["aud"] != GATEWAY or claims["exp"] <= now:
            return 401, {"error": {"code": -32001, "message": "bad token"}}, None
        if headers.get("mcp-method") != method:
            return 400, {"error": {"code": -32020, "message": "header mismatch"}}, None
        if name is not None and headers.get("mcp-name") != name:
            return 400, {"error": {"code": -32020, "message": "header mismatch"}}, None
        if method != "tools/call":
            return None, None, None
        if name not in self.allow:
            return 403, {"error": {"code": -32001, "message": "tool not allowed"}}, None
        key = (user, name)
        recent = [t for t in self.hits.get(key, []) if now - t < self.window]
        if len(recent) >= self.limit:
            return 429, {"error": {"code": -32001, "message": "rate limited"}}, None
        self.hits[key] = recent + [now]
        return None, None, None


def request_token(headers):
    return headers.get("authorization", "").replace("Bearer ", "")


def call(user, tool, token=None, arguments=None, headers_name=None):
    """A tools/call as the host would send it, with the mirrored headers."""
    msg = {"jsonrpc": "2.0", "id": 1, "method": "tools/call",
           "params": {"name": tool, "arguments": arguments or {}}}
    headers = {"authorization": "Bearer " + (token or issue(GATEWAY, user)),
               "mcp-method": "tools/call", "mcp-name": headers_name or tool}
    return {"user": user, "headers": headers, "body": msg}
gw = Gateway([make_copy(1), make_copy(2)], limit=3,
             allow={"get_campaign", "search_docs"})
now = 1000
asked = [("ann", "get_campaign"), ("ann", "get_campaign"), ("bob", "search_docs"),
         ("ann", "get_campaign"), ("ann", "get_campaign"), ("ann", "delete_campaign")]
for i, (user, tool) in enumerate(asked):
    status, body = gw.handle(call(user, tool), now + i)
    print(user.ljust(4), tool.ljust(16), status)
mismatch = call("ann", "get_campaign", headers_name="search_docs")
print("header mismatch:", gw.handle(mismatch, now + 10)[0])
upstream = call("ann", "get_campaign", token=issue(UPSTREAM, "ann"))
print("wrong audience: ", gw.handle(upstream, now + 11)[0])
Worked example

Read the audit log

Every request left an audit line, including the refused ones. This cell sends a day's traffic through the gateway, then counts calls and errors per tool and finds the slowest call, then draws it.

Uses the setup above, 92 lines.

from collections import Counter

import matplotlib.pyplot as plt

gw = Gateway([make_copy(1), make_copy(2)], limit=100,
             allow={"get_campaign", "search_docs", "export_report"})
tools = ["get_campaign", "search_docs", "get_campaign", "export_report"]
for i in range(16):
    tool = tools[i % 4]
    fail = tool == "export_report" and i % 8 == 3
    gw.handle(call(["ann", "bob"][i % 2], tool, arguments={"fail": fail}), 1000 + i)
gw.handle(call("ann", "delete_campaign"), 1020)

calls = Counter(line["tool"] for line in gw.audit)
errors = Counter(line["tool"] for line in gw.audit if line["error"])
slowest = max(gw.audit, key=lambda line: line["ms"])
names = sorted(calls)
print({name: (calls[name], errors[name]) for name in names})
print("slowest:", slowest["tool"], slowest["ms"], "ms, copy", slowest["copy"])

fig, (left, right) = plt.subplots(1, 2, figsize=(9, 3.2))
left.bar([n for n in names], [calls[n] for n in names], label="calls")
left.bar([n for n in names], [errors[n] for n in names], label="errors")
left.set_title("Calls and errors by tool")
left.set_ylabel("requests")
left.legend()
right.bar(names, [max(l["ms"] for l in gw.audit if l["tool"] == n) for n in names])
right.set_title("Slowest call by tool")
right.set_ylabel("milliseconds")
for axis in (left, right):
    axis.tick_params(axis="x", labelrotation=20)
plt.tight_layout()
plt.show()
In the lesson you can change this and run it.

Four numbers answer most production questions: how many calls each tool gets, how many fail, which is slowest and which copy answered. export_report is both the slowest and the one that fails, and delete_campaign was refused before it reached a server, so it shows as an error in the log. Nothing here needed the request bodies.

Quick check
In the audit chart's data, delete_campaign appears as one error and no successful call. Why?
  1. The server crashed while running the tool
  2. The rate limit was reached for that user
  3. The model chose a tool that doesn't exist
  4. The gateway refused it with 403: not on the allowlist
Show the answer

The gateway refused it with 403: not on the allowlist

The allowlist doesn't include delete_campaign, so the gateway answered 403 itself. The refusal is logged like any other request, which is how an audit finds attempts, not only successes.

Quick check
The gateway forwards 5 requests to two copies in turn, starting with copy 0. How many does copy 0 get?
copies = 2
served = [turn % copies for turn in range(5)]
print(served.count(0))
  1. 3
  2. 2
  3. 5
  4. 1
Show the answer

3

The turns go 0, 1, 0, 1, 0, so copy 0 gets requests 1, 3 and 5. Round robin is even over many requests and off by one over an odd number.

Drill

How long until a slot frees up?

Write retry_after(hits, now, window, limit): hits are the times of recent accepted calls. Return 0 if fewer than limit of them are inside the window, otherwise the whole seconds until the oldest hit in the window leaves it.

Uses the setup above, 92 lines.

def retry_after(hits, now, window, limit):
    return 0


print(retry_after([10, 20, 30], 40, 60, 3))
In the lesson, your code is checked as you write it.
Drill

Which tools are slowest?

Write slowest(audit, k): audit is a list of dicts with tool and ms. Return the names of the k tools with the largest single call, slowest first. Each tool appears once.

Uses the setup above, 92 lines.

def slowest(audit, k):
    return []


log = [{"tool": "a", "ms": 40}, {"tool": "b", "ms": 900}, {"tool": "a", "ms": 120}]
print(slowest(log, 2))
In the lesson, your code is checked as you write it.
In your own words
The audit log shows export_report is the slowest tool and fails most. What do you check next, and where?

Write a sentence or two, then compare it with a good answer in the lesson.

In production: traces, ownership and what goes wrong

Purpose. Connect the audit line to the rest of the system, and say who owns what.

An audit line tells you a call was slow. A trace tells you why. The 2026-07-28 revision documents OpenTelemetry trace context in _meta: a request may carry traceparent (and tracestate, baggage), so one trace id follows a request from the host through the gateway and the server to the APIs the server calls. The gateway copies the id into its audit line, so the log and the trace join on it.

What a platform team owns. The token settings (audience and scopes), the allowlists per team, the rate limits and the retention of the audit log. Application teams own their tools. When a tool is added, the allowlist change is a reviewed change, and when a limit is hit, the 429 carries a Retry-After so a client waits instead of hammering.

In an engineer's week. Someone asks "who called the refund tool last Tuesday?" The audit log answers in a minute if every request, refused ones included, wrote a line with the user, the tool and the status. What goes wrong is a gateway that logs only successes, a rate limit shared across all users so one script locks out everyone, and a gateway that forwards the user's token because it was easier.

Goes deeper in

Quick check
The limit is 5 calls in 60 seconds, and the oldest of a user's 5 hits was 40 seconds ago. What Retry-After does a 6th call get?
  1. 40
  2. 60
  3. 0
  4. 20
Show the answer

20

Retry-After is the time until the oldest hit leaves the window: 60 − 40 = 20 seconds. Then there is room for one more call.

In your own words
A single script is hammering one tool and other users are getting 429. What was wrong with the rate limit, and what should it be?

Write a sentence or two, then compare it with a good answer in the lesson.

Practice what you just read

  • 2 code examples you can edit and run
  • a practice exercise that checks your code
  • a 5-question quiz
  • 9 review cards for spaced repetition

Try a quiz question

Why can a gateway enforce a per-tool allowlist without parsing the request body?

  1. The tool name is repeated in the Mcp-Name header
  2. The gateway guesses the tool from the URL path
  3. Tool names are always inside the access token
  4. It cannot: it has to parse the whole body first

Answer it at the end of the lesson to see if you're right, and why.

Before this lesson

Keep learning

Where this lesson leads.

575 lessons in thirteen phases. 176 are free without an account, and a free account opens the rest. See every lesson.

Phase 9 · about 13 hours of study

Model Context Protocol

Build MCP servers, clients and hosts, from the wire format to production 17 lessons 1 free
  1. JSON-RPC 2.0 in depth FreeRequests, notifications, errors and a dispatcher built from scratch.35 min
  2. Versions and stateless requestsRevisions by date, the legacy handshake, per-request _meta and version negotiation.30 min
  3. Discovery, capabilities and the two erasserver/discover, declared capabilities, and talking to legacy and modern software alike.30 min
  4. Transports I: stdioNewline-delimited JSON over a subprocess's standard streams.30 min
  5. Transports II: Streamable HTTPOne endpoint, required headers, JSON or event-stream replies.40 min
  6. Build a server: toolsTool definitions, schemas from type hints, results and two kinds of error.45 min
  7. Resources and promptsReadable data, URI templates and reusable prompt templates.40 min
  8. The same server with the official SDKsYour registry and dispatcher on the SDK's decorator shape, in Python and TypeScript.40 min
  9. Build a clientMetadata on every request, version fallback, paging and caching.40 min
  10. Hosts: connecting servers to a modelMany servers, one agent: namespacing, routing and approvals.40 min
  11. Progress, subscriptions and multi-round-trip requestsLong-running work, change notifications and asking the user mid-call.45 min
  12. Securing remote servers with OAuthProtected resource metadata, PKCE, audiences and scope challenges.45 min
  13. MCP securityTool poisoning, rug pulls, confused deputies and local-server risks.40 min
  14. Lab: attack and defend a hostPoisoned tools, injected results, token replay and the defenses that stop each.40 min
  15. Testing, designing and shipping serversConformance checks, tool design for models, SDKs and deployment.40 min
  16. MCP in production: gateways, limits and auditAn allowlist, per-user rate limits, header checks and an audit log in front of many servers.35 min
  17. Capstone: a production MCP serverA complete, tested server for the practice shop.45 min

Never trained an AI? Teach a small one to write, right in your browser. It takes about 20 seconds and needs no account. Teach it to write