AI agents are not short on intelligence. What they lack is governed access to your IT systems.

Here is the reference architecture for an on-premise MCP gateway, split into independent planes, and what it concretely changes for an IT department.

From spaghetti to hub

Before: three agents each wired to three tools, nine integrations. After: one central MCP gateway, six connections

Take an industrial group that connects Claude, Copilot and an HR agent to its CRM, ERP and document management system. Without a hub, every team wires its own integrations, with its own secrets. With the gateway, each tool is connected once, each agent once.

Key benefit: a fourth agent costs one connection, not three.

Three decoupled planes

Control plane, data plane and observability plane, kept separate and linked by configuration and telemetry

  • Control plane: registry, policies and identity are versioned in Git. Removing the delete_customer tool for the whole company = one merge request, no redeployment.
  • Data plane: the gateway is stateless. When usage grows, Kubernetes adds replicas; if one fails, no agent is cut off.
  • Observability plane: telemetry is sent asynchronously. If monitoring goes down, calls keep flowing.

Anatomy of a call

A sales rep asks Claude: “Where is order 4512?”

Call flow: Claude, MCP gateway, identity provider and ERP, in eight steps down to the audit trail

The model never sees the ERP token, and the ERP sees the sales rep’s real identity: her existing permissions apply, with nothing rewritten.

Zero trust, tool by tool

Profile Visible tools Human approval
Sales get_order, search_customer, create_quote send_quote (sent to the client)
Support get_ticket, get_order refund_order
HR search_employee, get_leave_balance Payroll tools hidden

Each profile only sees its own virtual server: a shorter context for the model, a smaller attack surface.

End-to-end observability

The gateway, MCP servers and APIs send telemetry to an OpenTelemetry collector, which feeds traces, metrics and audit

Incident: support reports slowness. The trace shows the latency comes from the ERP API, not the gateway or the model. The ticket goes straight to the right team.

Audit: “who triggered a refund through an agent this month?” is answered with one query in the SIEM.

Key strengths at a glance

Enterprise requirement How the architecture delivers
Sovereignty On-premise: tokens, parameters and responses stay inside your IT
Security OIDC, per-tool RBAC, token exchange, secrets in a vault
Scalability Stateless data plane, horizontal scaling
Agility Policies versioned in Git, no redeployment
Reuse Existing REST APIs exposed through OpenAPI, no rewrite
Auditability OpenTelemetry traces and an audit log sent to the SIEM
Reversibility Open standards (MCP, OpenTelemetry): no vendor lock-in

Getting started

If the company already runs an API gateway (Kong, Apigee, Azure API Management), switch on its MCP features. If not, IBM ContextForge, which is open source, makes for a fast proof of concept.

  1. Pilot: two or three high-value APIs, one team, the IdP connected.
  2. Industrialisation: Kubernetes, policies in Git, a dashboard per tool.
  3. Roll-out: a catalogue open to all teams, new tools added through self-service.

The gateway is the front door of any agentic architecture. The real topic is not AI, it is architecture: that is what turns a prototype into a product.