MCP server architecture basics

Published 22 Mar 2026 · By Adam C, Core Dynamics

At Core Dynamics, clients almost never start by saying, “We need MCP.” They usually say something more direct: “We want the assistant to read our systems, maybe write later, and we need to know it will not break anything.” That is the real brief. By the time teams call us, they are usually done with flashy demos. They want control.

We have been in kickoff calls where the first notes on the whiteboard were not speed or model choice. They were simpler than that. Who can call what? What gets logged? What happens when a backend fails? That is where good MCP architecture starts.

Model Context Protocol, or MCP, gives AI clients a standard way to find tools, call them, and exchange structured context with external systems. In plain terms, an MCP server is the controlled middle layer between an AI client and the systems a business actually runs on: APIs, databases, workflows, documents, and internal services.

The pattern is pretty consistent. Clients want read access before write access. They want a short list of solid tools before a giant catalog. And they want every call to be easy to trace. So for engineers, the job is to expose useful capability without exposing chaos. For everyone else, the simple version is this: the MCP server is the doorway, the guardrail, and the record of what happened.

What an MCP server actually is

An MCP server sits between an AI client and one or more backend systems. The client might be a desktop assistant, a browser agent, an internal ops console, or another service. The server publishes tools and schemas. The client discovers those tools, sends structured calls, and gets structured responses back.

Here is the plain-English version. If a chatbot needs to look up a customer invoice, it should not talk straight to the accounting database. It should talk to an MCP server instead. The server exposes a tool like get_invoice(invoice_id), checks the input, applies access rules, calls the right backend, and returns only what the client is allowed to see.

The core architectural layers

A solid MCP server usually has five layers. Early prototypes often mash them together, and that is fine for a week or two. But once real users show up, the separation matters.

1. Transport layer

This is the front door. It handles incoming MCP requests, stream mechanics, content types, and response framing. It should not hold business logic. Its job is to translate protocol traffic into internal application calls.

2. Tool registry and schemas

This layer defines what tools exist, what they are called, what arguments they accept, and what shape they return. Tight schemas matter. Models behave better when the contract is narrow and clear instead of vague and open-ended.

3. Orchestration layer

This is where routing, retries, policy checks, and cross-service composition happen. One tool might call a single backend. Another might touch three systems and stitch the answer together. This layer is where most of the real design work lives.

4. Integration adapters

Adapters connect the server to databases, CRM APIs, file stores, search indexes, or internal services. Keeping them separate stops protocol logic from getting tangled up with vendor SDK code.

5. Observability and policy controls

Production MCP servers need logs, traces, error handling, rate limits, and auth checks. If an AI client can call a tool, you should be able to answer a few basic questions fast: who called it, what they passed in, which system it touched, and what came back.

What happens during a request

The easiest way to understand MCP is to follow one request from start to finish.

1. Discovery: the client learns which tools the server exposes and what input schema each tool expects.
2. Selection: the language model decides that a tool is needed for the current task.
3. Invocation: the client sends a structured tool call to the MCP server.
4. Validation: the server verifies required fields, types, size limits, and policy constraints.
5. Execution: the orchestration layer invokes the relevant adapter or internal service.
6. Normalization: raw backend output is transformed into a predictable, model-friendly response.
7. Return: the client receives the result and uses it in the next reasoning step.

Here's the thing: an MCP server is not just an API wrapper. It is the layer that turns a fuzzy model decision into a clean, deterministic action. If that layer is weak, the whole system gets weird fast.

That is also why “the AI has access to our systems” is the wrong mental model. What matters is that the AI has access only through a small set of server-defined operations, each with a tight contract and a traceable path.

Security and trust boundaries

Most MCP architecture mistakes are really trust-boundary mistakes. Teams assume the model will behave if the tool descriptions are clear enough. That is not a security plan. The server should act as if every call could be malformed, too broad, replayed, or made by a client with the wrong permissions.

At a minimum, the server should enforce authentication, authorization, input validation, output filtering, and rate limits. Sensitive tools should sit in tighter lanes. A read-only lookup tool should not live in the same trust tier as a write action that can change records, trigger payments, or open tickets.

We also see teams return too much data because it feels convenient. But convenience is not the goal. If a billing tool needs only invoice status and due date, it should not also return payment history, internal notes, and account IDs unless those fields are clearly required.

Patterns that matter in production

Once a prototype touches real users, three patterns matter almost every time.

Thin tools, thick policies

Keep tool interfaces simple, but make policy enforcement deep. A tool should feel easy to call, while the server quietly handles validation, auth checks, idempotency, and backend-specific safeguards.

Stable contracts, replaceable adapters

The client-facing tool contract should stay stable even if the backend changes. That way, you can move from one CRM vendor to another, or from a direct SQL query to an internal service, without retraining users or rewriting the whole client flow.

Trace everything that affects outcomes

Production incidents rarely start in one clean place. They come from a chain: the client picked a tool, passed weak context, the server retried a downstream call, and the result came back half right. Without traces and structured logs, root-cause work turns into guesswork.

A good MCP server should feel boring on a bad day. That is a compliment. It means the system is predictable when things go sideways.

Architecture diagrams

These three views are the ones we use most when we explain MCP projects to clients. First, the structure. Then the request flow. Then the trust boundaries. In our experience, that sequence helps both engineering teams and business stakeholders get to the same picture faster.

Diagram 1: high-level MCP architecture

Tool call lifecycle

Trust boundaries and security zones

Final point

MCP matters because it forces teams to be clear. Clear tool boundaries. Clear schemas. Clear permissions. Clear execution paths. That is what turns an agent system from a demo into something a real team can trust.

At Core Dynamics, that is what clients ask us for most once the first prototype hits real systems. Fewer surprises. Tighter boundaries. Better logs. Tools that keep working after the demo video is over. It is rarely about adding more tools. It is usually about making the right tools behave well.

If I had to boil it down to one rule, it would be this: keep the tool surface small, keep the policy layer strict, and keep the logs readable. Do that, and MCP stays useful. Skip it, and the architecture can look fine right up until it does not.