AI agents are not short on intelligence. What they lack is governed access to your IT systems.
Here is the reference architecture for an on-premise MCP gateway, split into independent planes, and what it concretely changes for an IT department.
From spaghetti to hub
Take an industrial group that connects Claude, Copilot and an HR agent to its CRM, ERP and document management system. Without a hub, every team wires its own integrations, with its own secrets. With the gateway, each tool is connected once, each agent once.
Key benefit: a fourth agent costs one connection, not three.
Three decoupled planes
- Control plane: registry, policies and identity are versioned in Git. Removing the
delete_customertool for the whole company = one merge request, no redeployment. - Data plane: the gateway is stateless. When usage grows, Kubernetes adds replicas; if one fails, no agent is cut off.
- Observability plane: telemetry is sent asynchronously. If monitoring goes down, calls keep flowing.
Anatomy of a call
A sales rep asks Claude: “Where is order 4512?”
The model never sees the ERP token, and the ERP sees the sales rep’s real identity: her existing permissions apply, with nothing rewritten.
Zero trust, tool by tool
| Profile | Visible tools | Human approval |
|---|---|---|
| Sales | get_order, search_customer, create_quote |
send_quote (sent to the client) |
| Support | get_ticket, get_order |
refund_order |
| HR | search_employee, get_leave_balance |
Payroll tools hidden |
Each profile only sees its own virtual server: a shorter context for the model, a smaller attack surface.
End-to-end observability
Incident: support reports slowness. The trace shows the latency comes from the ERP API, not the gateway or the model. The ticket goes straight to the right team.
Audit: “who triggered a refund through an agent this month?” is answered with one query in the SIEM.
Key strengths at a glance
| Enterprise requirement | How the architecture delivers |
|---|---|
| Sovereignty | On-premise: tokens, parameters and responses stay inside your IT |
| Security | OIDC, per-tool RBAC, token exchange, secrets in a vault |
| Scalability | Stateless data plane, horizontal scaling |
| Agility | Policies versioned in Git, no redeployment |
| Reuse | Existing REST APIs exposed through OpenAPI, no rewrite |
| Auditability | OpenTelemetry traces and an audit log sent to the SIEM |
| Reversibility | Open standards (MCP, OpenTelemetry): no vendor lock-in |
Getting started
If the company already runs an API gateway (Kong, Apigee, Azure API Management), switch on its MCP features. If not, IBM ContextForge, which is open source, makes for a fast proof of concept.
- Pilot: two or three high-value APIs, one team, the IdP connected.
- Industrialisation: Kubernetes, policies in Git, a dashboard per tool.
- Roll-out: a catalogue open to all teams, new tools added through self-service.
The gateway is the front door of any agentic architecture. The real topic is not AI, it is architecture: that is what turns a prototype into a product.