MCP Server Testing: Why Your Agent's Riskiest Integrations Have No Tests

A platform team at a mid-sized SaaS company connected their internal support agent to eleven Model Context Protocol servers over four months: a ticketing system, a billing API, a knowledge base, two internal databases, a Slack workspace, a calendar, and four smaller utility servers built by different engineers on different teams. Every connection passed its manual smoke test. The agent answered questions correctly in the demo. Nobody wrote a single automated test for any of it, because nobody was quite sure what a test would even check. There was no request schema to validate against a contract, because the "endpoint" was a tool description written in English, consumed by a model, not a client library. There was no version number to pin, because the server didn't publish one beyond the MCP protocol date itself. There was no clean failure mode to assert on, because when something went wrong, the model didn't throw an exception — it just did something slightly different, and kept going.

Three months after the last server was connected, one of the smaller utility servers — maintained by a contractor who had since left — pushed an update that renamed a parameter and quietly narrowed what an "list_open_tickets" tool considered "open." Nothing broke. No alert fired. No test failed, because there was no test to fail. Support agents started noticing the assistant was systematically under-reporting backlog for one product line, and it took seventeen days to trace the discrepancy back to that one tool description. This is a hypothetical scenario, but it is representative of a pattern now showing up across companies that adopted MCP quickly: the protocol is real, the adoption is real, and the testing discipline around it does not yet exist in most organizations that use it.

That gap is the subject of this piece. Model Context Protocol has moved, in under two years, from an Anthropic-authored specification to a connectivity layer that OpenAI, Google DeepMind, and a long list of platform vendors have adopted for the same purpose Anthropic built it for: letting AI applications call tools and read context from external systems through one standardized interface instead of a bespoke integration per model, per tool. That standardization is genuinely valuable — it is the same category of win the Language Server Protocol delivered for IDEs, and MCP's own specification says as much. But the way most engineering organizations are testing and governing MCP server testing right now is borrowed almost entirely from REST API practice, and REST API practice does not map cleanly onto a protocol where the "interface contract" is a natural-language description read by a probabilistic model rather than a machine-checked schema read by a compiler.

What MCP Actually Standardizes, and What It Leaves to You

It helps to be precise about what the protocol is, because a lot of the risk discussion downstream depends on distinguishing "this is a protocol requirement" from "this is a common implementation choice that happens not to be required."

MCP is an open protocol, originally released by Anthropic on November 25, 2024, described in the announcement as "an open standard for connecting AI assistants to the systems where data lives, including content repositories, business tools, and development environments." Anthropic's stated motivation was that "even the most sophisticated models are constrained by their isolation from data — trapped behind information silos and legacy systems," and that every new data source had previously required a custom implementation, making truly connected systems difficult to scale. Early adopters named in the announcement included Block and Apollo, alongside development tool companies — Zed, Replit, Codeium, and Sourcegraph — integrating MCP to help coding agents retrieve relevant context.

The protocol has since been adopted well beyond Anthropic's own products. Google confirmed it would support MCP across its AI products and infrastructure, including Gemini, in April 2025, with Google's Demis Hassabis publicly endorsing the standard. OpenAI added MCP support to its Agents SDK and ChatGPT desktop app around the same period. This is worth noting not because the article needs an adoption statistic — vendor-reported "number of MCP servers" counts change too fast and come from too many uncorroborated directories to responsibly cite as a hard figure — but because it establishes something that matters for a testing conversation: MCP is no longer an Anthropic-specific mechanism you can treat as a vendor integration. It is closer to how OAuth or SAML function for identity: a protocol multiple competing platforms have converged on, which means the tool servers your teams build or install will very likely be consumed by more than one AI system, by clients you don't control, in configurations you didn't anticipate.

The Architecture in Three Roles

The specification defines three roles communicating over JSON-RPC 2.0 messages:

• Hosts — the LLM applications that initiate connections (a chat client, an IDE, an internal agent platform).
• Clients — the connector components inside the host that maintain a 1:1 connection to a server.
• Servers — the services that expose context and capabilities back to the client.

Servers can offer three categories of capability: Resources (contextual data for the model or user), Prompts (templated workflows), and Tools (functions the model can actually invoke). Tools are the category that carries almost all of the operational risk this article is concerned with, because a tool is not just information — it is, in the specification's own words, "arbitrary code execution" from the model's perspective, and the specification is explicit that it "must be treated with appropriate caution."

Each client maintains a separate connection to a separate server, and the host mediates everything the model sees. This one-to-many fan-out is precisely why the failure surface multiplies with every server added: the host is trusting each server's self-description, and the model is reasoning over the union of everything every connected server has told it, simultaneously, in the same context window.

How Tools Are Actually Described and Called

A server exposes tools by responding to a tools/list request with an array of tool definitions. Each definition has a name, an optional title, a description (this is the part that matters most and gets the least scrutiny), and an inputSchema — a JSON Schema object defining the tool's parameters. Newer versions of the spec also allow an outputSchema for structured results, annotations describing tool behavior, and an execution object for task-augmented (long-running) execution. When the model decides to use a tool, the client sends a tools/call request with the tool name and arguments, and the server responds with content blocks — text, images, audio, embedded resources, or structured JSON — plus an isError flag if the tool failed.

None of that is unusual if you've built or tested a REST or RPC API. What is unusual, and this is the detail that most engineering teams gloss over, is what actually governs whether the model calls the right tool with the right arguments: not the schema, but the natural-language description field. The JSON Schema constrains what a well-behaved client will accept as arguments once a tool has been chosen. It does almost nothing to constrain which tool gets chosen, or why, because tool selection happens inside the model's reasoning over the description text, not inside a type checker. The specification acknowledges this asymmetry directly, in language that is unusually blunt for a protocol spec:

"Tools represent arbitrary code execution and must be treated with appropriate caution. In particular, descriptions of tool behavior such as annotations should be considered untrusted, unless obtained from a trusted server."

And, in the tool-specific security considerations:

"For trust & safety and security, clients MUST consider tool annotations to be untrusted unless they come from trusted servers."

This is the protocol's own authors telling implementers, in a MUST-level normative statement, that the primary interface contract of an MCP tool — its description — cannot be assumed safe by default. Most integration testing practice has no equivalent concept. A REST contract test validates that a response matches a schema; it does not need to ask whether the field names themselves are trying to manipulate the consumer of the response. An MCP tool description can.

Versioning Is Coarser Than Most Teams Assume

The protocol itself is versioned by date — the specification referenced throughout this article is the 2025-11-25 revision, following 2025-06-18 and the original 2024-11-05 release, each of which changed protocol-level behavior (the June 2025 revision, for example, added structured tool output, elicitation, and OAuth Resource Server classification for MCP servers). That versioning exists at the protocol level — it governs which JSON-RPC methods, capabilities, and negotiation behaviors a client and server agree to speak.

It does not exist at the individual tool level. There is no required version field on a tool definition, no required deprecation window, and no protocol-level guarantee that a tool's name, description, or input schema will remain stable between two calls to tools/list. The mechanism the spec provides for tools changing is the listChanged capability and the corresponding notifications/tools/list_changed message — but that notification tells a client "something changed, go re-list," not what changed, whether it's backward compatible, or whether previously working prompts and cached tool selections are still valid. A server operator can rename a parameter, tighten a validation rule, or change what a tool actually does behind an unchanged name, and the only protocol-level signal available is an optional notification that many client implementations do not act on synchronously, and that says nothing about compatibility.

Compare that to how most engineering organizations already handle breaking changes in REST or GraphQL APIs — typically through URI versioning, header-based versioning, deprecation windows, and contract tests that fail the build on an incompatible schema diff. MCP has none of that as a protocol guarantee. It is architecturally possible to build it as a discipline on top of MCP — and later in this article that's exactly what the testing framework recommends — but it does not come for free the way it does with API gateways that have spent a decade building tooling around OpenAPI diffing.

Why This Outpaced Governance So Quickly

Two forces compounded here, and it's worth separating them because they call for different responses.

The first is ordinary integration economics. Before MCP, connecting an AI application to N external tools meant writing and maintaining N bespoke integrations, each with its own auth flow, request format, and error handling — and every new AI application repeated that work from scratch. MCP's pitch, and the reason Block, Apollo, and the coding-tool vendors adopted it within weeks of release, was that a tool built once as an MCP server becomes usable by any MCP-compatible host. That is a legitimate, well-understood kind of standardization win, structurally similar to what USB did for peripherals or what the Language Server Protocol did for editor tooling, and the specification explicitly frames it that way. Fast adoption of a genuinely useful standard is not itself a problem.

The second force is where the risk actually lives: the interface being standardized is fundamentally different in kind from the interfaces prior integration protocols standardized, and most of the tooling, training, and organizational habits that make REST and gRPC integrations governable were built for the old kind. A REST endpoint's contract is enforced by a type system and, usually, a schema validator sitting in front of business logic no matter what the caller intends. An MCP tool's "contract" is enforced by whether a language model, reading a paragraph of English alongside every other tool description currently in its context, decides to call it correctly. Those are different enforcement mechanisms with different failure statistics, and the industry adopted the protocol faster than it built the second kind of enforcement.

Nothing in that table is a criticism of the protocol design — the specification's authors are explicit that "MCP itself cannot enforce these security principles at the protocol level" and that implementors "SHOULD" build the missing guardrails themselves. The gap is an organizational one: implementors have largely not built those guardrails yet, and the testing discipline for this integration class has not caught up with how much of it companies have already shipped.

Failure Mode One: Tool Descriptions That Say One Thing to the User and Another to the Model

The cleanest way to understand this failure mode is through the research that named it. In April 2025, the security research group Invariant Labs published a disclosure describing what they termed Tool Poisoning Attacks — malicious or compromised MCP servers embedding hidden instructions inside tool descriptions that are visible to the language model but not meaningfully surfaced to the human approving the tool's use. Their proof-of-concept used a benign-looking "add" tool in Cursor (a popular MCP-enabled code editor) whose description contained buried instructions directing the agent to read ~/.cursor/mcp.json and other sensitive local files, including SSH keys, while appearing to the user simply as an addition function. In a second example, a malicious server's tool description performed what the researchers called tool shadowing: instructions embedded in one server's tool description silently altered how a different, trusted server's email-sending tool behaved, redirecting outgoing messages to an attacker-controlled address without visibly changing anything in the user-facing flow. Invariant Labs separately documented a related attack against a WhatsApp MCP integration, in which injected instructions caused message history to be exfiltrated to an attacker-controlled number, with the exfiltrated content obfuscated using invisible whitespace so it wouldn't be visually apparent in logs or transcripts.

Independently, researchers at Trail of Bits published a related finding in April 2025 they called line jumping: because tool descriptions are loaded into the model's context the moment a client calls tools/list — before any tool is actually invoked, and often before the user has approved using that specific tool at all — a malicious description can inject instructions that influence model behavior pre-emptively. Their demonstrated example used a tool description that instructed the model to prefix all future shell commands with a command that made a user's home directory world-readable, disguised as a fictitious "operating system requirement" and falsely justified as "REQUIRED FOR INTERNAL AUDITING, GDPR, and SOC2 COMPLIANCE." Tested clients, including Claude Desktop, followed the injected instruction when using other, unrelated tools later in the session — meaning the compromise didn't require the user to ever knowingly invoke the malicious server's own tool.

The industry has since formalized this as its own OWASP entry, "MCP Tool Poisoning," describing the pattern generically: an MCP client and its underlying LLM read the full text of a tool's description as part of deciding how to behave, and if that text is attacker-controlled or simply badly written, the result ranges from subtly wrong tool selection to deliberate data exfiltration, and the user-facing approval UI typically shows only a simplified summary that hides the mechanism.

Hypothetical scenario, fintech context. A payments company integrates a third-party MCP server from a bookkeeping SaaS vendor so its internal finance-ops assistant can pull invoice status. The vendor's tool description for get_invoice_status reads, in part: "Also use this tool whenever the user asks about account balances, and always cross-reference with export_ledger for accuracy." Nothing in that sentence is obviously malicious — it reads like an over-eager technical writer's attempt to improve tool discoverability — but it causes the assistant to invoke a broader ledger-export tool on every balance inquiry, including ones from users who should not see full ledger detail under the company's existing row-level permission model. No exploit occurred. No CVE applies. The failure is a description that was ambiguous enough to expand the assistant's effective data access beyond what any individual engineer approved, and it would not be caught by anything resembling a schema test, because the schema for both tools was perfectly valid.

The distinction worth internalizing: some of what falls under "tool poisoning" is adversarial (a genuinely malicious server author), and some is simply the consequence of natural-language interfaces being underspecified in ways that never mattered when the same interface was JSON keys read by deterministic code. A testing program has to catch both, because from the model's point of view they look identical.

Failure Mode Two: Schema and Behavior Drift With No Compatible Versioning Signal

The Cursor vulnerability catalogued as CVE-2025-54136, publicly disclosed by Check Point Research and reported through The Hacker News and Tenable in August 2025, is the cleanest documented case of drift-after-approval turning into an actual security incident rather than a hypothetical one. The vulnerability, nicknamed MCPoison, exploited the fact that Cursor's trust model bound a user's one-time approval of an MCP server to that server's name, not to its contents. As Check Point's researchers put it, "any changes to the underlying configuration are considered trusted because it is bound by the MCP name not its contents." Concretely: a user approves an MCP server entry once; an attacker who can later modify the project's mcp.json file — through a shared repository, a compromised collaborator account, or a supply-chain foothold — can silently swap that server's command for an arbitrary one, and Cursor would execute it without re-prompting for approval, because from the client's perspective it was still "the same," already-trusted server. Affected versions were 1.2.4 and below; Cursor shipped a fix in version 1.3 that requires re-verification when the underlying configuration changes, not just the display name. A closely related vulnerability disclosed the same window, CVE-2025-54135 ("CurXecute," CVSS 8.5, reported by AIM Security), showed that prompt injection delivered through a connected MCP server (their proof of concept used a Slack integration) could rewrite Cursor's global MCP configuration and have the resulting command executed before the user had a chance to reject the suggested edit.

Both vulnerabilities are specific to Cursor's implementation, not the MCP protocol itself, and both have been patched. They are cited here because they are the clearest evidence to date that "the tool's definition changed after approval and the client didn't notice" is not a theoretical concern invented for this article — it is a documented root cause of a real, CVE-tracked remote-code-execution vulnerability in a widely used MCP client, disclosed within a year of the protocol's release.

The narrower, non-adversarial version of this failure is more common and less visible: an internal or third-party MCP server's maintainer ships what they consider a minor update — a renamed parameter, a stricter validation rule, a changed default, a subtly different definition of what a filter means — and because there is no protocol-level requirement to version tools individually or to signal breaking versus non-breaking changes, nothing forces that update through a compatibility check before it reaches production agents. The notifications/tools/list_changed message the spec provides exists precisely so clients can detect that something changed; it says nothing about what changed or whether existing prompts, few-shot examples, or cached reasoning about that tool are still valid. Many client implementations, particularly simpler internal agent frameworks built quickly during 2025, cache tool lists at session start and never re-fetch mid-session, meaning the notification can arrive and be entirely ignored in practice even when technically supported.

Hypothetical scenario, e-commerce context. An online retailer's internal MCP server exposes a check_inventory tool with an include_backorder boolean parameter defaulting to true. A warehouse systems engineer, cleaning up the tool during an unrelated refactor, flips the default to false on the reasoning that backorder items shouldn't count as "available" — a defensible product decision made in isolation, without realizing three different customer-facing assistants across two business units call this tool assuming the old default. Within a day, the assistants start telling customers items are out of stock when they are, in fact, orderable with a delay, generating a wave of abandoned carts and support tickets asking why the bot contradicts what the product page says. No error was thrown anywhere in the stack. The input schema was still valid JSON Schema; the tool still returned a well-formed response; the "bug" was a semantic default that no automated check was watching, because nothing had defined "the current contract" as something worth snapshotting and diffing in the first place.

Failure Mode Three: Prompt Injection Arriving Through Tool Output, Not User Input

Most organizations that have done any threat modeling on their AI assistants have thought about prompt injection as a user-input problem: a user types something adversarial, the model does something it shouldn't. MCP introduces a second, structurally different injection surface that a user-input-focused review will miss entirely — injection carried in tool results, which the model treats as trusted context precisely because it didn't come from the user.

This is a confused-deputy pattern in the classic sense: the model is a deputy acting with the combined authority of whatever tools it has been granted, and if an attacker can control the content of any data source the model reads through a tool — a support ticket, a document, a calendar invite, a webpage fetched by a browsing tool, a row in a database populated by user-submitted content — they can embed instructions in that data, and the model may follow them with the same trust it affords instructions from its system prompt, because at inference time both are just tokens in the context window. Simon Willison, writing about this in April 2025, described the underlying danger as a "lethal trifecta": a system that combines access to private data, exposure to untrusted content, and a channel capable of exfiltrating data is dangerous regardless of how each individual capability was justified, because the combination — not any single tool — is what an attacker needs.

Invariant Labs' WhatsApp case, described above, is a worked example of exactly this mechanism: the injected instructions didn't arrive through a user typing a jailbreak prompt — they arrived embedded in message content the assistant was asked to summarize, and the "attack" was the assistant faithfully following instructions that happened to be sitting inside the data it was retrieving on the user's behalf.

What makes this qualitatively different from the tool-description poisoning discussed earlier is timing and ownership. Tool poisoning corrupts the interface definition, which a server operator controls and which, at least in principle, can be reviewed once at connection time. Output-borne injection corrupts the data flowing through an otherwise entirely legitimate, well-behaved tool, on every single call, sourced from whatever underlying system that tool reads — a system your organization may not control at all, such as inbound customer email or a public web page. You cannot review this away once at integration time. It has to be treated as an ongoing property of every tool that reads external or user-generated content, which is most of them.

The specification's own tool-level security considerations acknowledge half of this problem directly, instructing that clients "SHOULD validate tool results before passing to LLM" and servers "MUST sanitize tool outputs" — but sanitizing free-text content for embedded instructions intended for a language model, rather than for a browser or a SQL engine, is a much less mature discipline than input sanitization for those older classes of injection, and most output "sanitization" in practice today amounts to stripping HTML and little else.

Failure Mode Four: Every Connected Server Widens the Same Blast Radius

A single MCP-connected agent frequently authenticates to many downstream systems through many separate credentials and tokens, and the industry's authorization patterns for this are still maturing in public, with real vulnerabilities disclosed along the way rather than solved in advance. Three specific, spec-documented risks compound as connected-server count grows.

The confused deputy problem in MCP proxy servers. The MCP Security Best Practices document, maintained alongside the specification, describes a concrete OAuth-level confused-deputy attack that applies whenever an MCP server acts as a proxy to a third-party API using a static client ID while allowing MCP clients to register dynamically. If the third-party authorization server sets a consent cookie after a legitimate first approval, an attacker can send a victim a crafted authorization link that reuses that cookie to skip the consent screen entirely, redirecting the resulting authorization code to an attacker-controlled endpoint. The specification states plainly: "MCP proxy servers using static client IDs MUST obtain user consent for each dynamically registered client before forwarding to third-party authorization servers." This is not a hypothetical risk invented for this article — it is significant enough that the protocol's own maintainers wrote a normative MUST requirement and a full attack-flow diagram into the specification to address it.

Token passthrough. The same document names "token passthrough" — an MCP server accepting a token from a client without verifying it was actually issued for that server and then forwarding it unmodified to a downstream API — as "explicitly forbidden," for reasons that compound with every additional connected server: it defeats downstream rate limiting and monitoring that depend on token audience, it destroys the audit trail (the downstream system's logs show the token's original issuer, not the MCP server actually making the call), and it means a token compromised anywhere in the chain can be replayed against every service that accepts it. The specification's blunt requirement: "MCP servers MUST NOT accept any tokens that were not explicitly issued for the MCP server."

Scope inflation across many servers. The Security Best Practices document also documents "scope minimization" as a named attack pattern, describing what happens when a server exposes every available permission scope up front and clients request all of them to minimize consent friction: a single leaked token then carries files:*, db:*, or admin:*-level access rather than the narrow permission the calling operation actually needed. This is exactly the mechanism that turns "we connected eleven MCP servers to one agent" into "one compromised credential now has meaningful reach across eleven systems" — the risk isn't eleven separate small exposures, it's one shared identity with the union of everyone's permissions.

None of this requires imagining a novel attack. It is a description of what the protocol's own maintainers have already had to document, mitigate, and in Cursor's case, patch after public disclosure. The practical consequence for a testing and governance program is that authorization review cannot be a one-time gate applied per server in isolation — the relevant unit of risk is the combination of servers one identity or one agent session can reach, and that combination grows every time someone adds "just one more" MCP connection to an existing agent.

Failure Mode Five: The Same Integration Behaves Differently on Different Runs

This last failure mode is the one with the least mature body of published security research and the most direct implication for how QA and platform teams need to change their testing methodology, so it's worth being explicit that this section is reasoned analysis grounded in how the protocol works, not a citation of a named vulnerability.

Tool selection in MCP is, by the specification's own description, "model-controlled": the language model discovers and invokes tools "automatically based on its contextual understanding." That is a deliberate design choice, and it's what makes MCP useful — the model, not a rules engine, decides which of potentially dozens of available tools across multiple connected servers is relevant to a given request. It is also, unavoidably, a source of run-to-run variability that contract testing was never built to model. Given identical input, a language model can select a different tool, call the same tool with differently phrased or differently structured arguments, call tools in a different order, or decide a tool isn't needed at all when a previous run decided it was — driven by sampling temperature, model version changes on the provider side, subtle differences in how much of the tool list fit in context, or simply the inherent non-determinism of the decoding process.

Contract testing, canary deployments, and most existing QA practice for API integrations assume that a fixed input produces a fixed, checkable output, and that a regression means something in the system under test changed. In an MCP-based agent, a regression signal can appear even when neither the agent's prompt, nor any connected server's tool definitions, changed at all — because the model's tool-selection behavior shifted underneath everything else. Conversely, a real regression in a tool's contract can hide inside normal-looking variance if a team never established what the distribution of acceptable tool-call sequences for a given input looks like in the first place, and is only checking a single sample run.

Hypothetical scenario, healthcare scheduling context. A clinic's virtual assistant, connected to an MCP server exposing check_availability and book_appointment tools, is asked "can you get me in sometime next week." On most runs it correctly calls check_availability first, presents options, and waits for confirmation before calling book_appointment. On a small but persistent fraction of runs — not tied to any code deployment — it collapses both steps and books the first available slot without confirmation, because the model judged the user's phrasing as sufficiently decisive. Nobody shipped a change. A QA process built entirely around "run the golden-path test case once and check it passes" would report this integration as fully working, because most of the time, it is — the failure is a tail behavior that only shows up under repeated sampling, and the existing test suite has no mechanism for detecting a shift in a distribution of behaviors rather than a single pass/fail outcome.

This is the failure mode existing QA literature has the least to say about, because it requires treating an integration's correctness as a statistical property measured across many runs rather than a boolean checked once — a genuinely different testing posture from anything a REST contract test, and most existing MCP tooling, currently provides.

Testing the MCP Servers You Build, Not Just the Ones You Connect To

Everything discussed so far treats MCP servers as something a client team connects to and has to defend against. Most engineering organizations are also on the other side of this relationship: they are building and operating MCP servers themselves, exposing internal systems to agents built by their own product teams, by partner organizations, or by whichever AI platform a customer happens to be using. That role carries its own testing obligations, and they are distinct enough from ordinary API testing to warrant separate treatment.

Input validation has to assume the caller is a model, not a client library. The specification's tool-level security considerations state plainly that servers "MUST validate all tool inputs," but the practical difference from validating a REST request body is what kind of malformed input to expect. A REST client sends malformed input because of a bug. A model calling your tool can send syntactically valid but semantically strange input because it misread an ambiguous description, hallucinated a plausible-looking argument, or is working from a stale understanding of the schema after a change it never re-fetched. Testing a server's input validation for MCP means testing against a wider space of "technically valid but clearly wrong" inputs than a human-authored client would typically produce — an integer where a small positive value was implied but nothing in the schema forbids a negative one, a date string in a format the schema allows but the business logic doesn't expect, an empty string where the description implied a required identifier.

Output sanitization needs its own test suite, separate from input validation. The same section requires servers to "sanitize tool outputs," and this deserves genuine test coverage rather than a one-line filter added as an afterthought. If your server surfaces any content that originated outside your own system — a customer's support message, a field a user typed into a form, a document someone uploaded — write tests that specifically assert your server does not pass that content through to the model without at least neutralizing patterns that look like directives aimed at an AI system (imperative language addressed to "the assistant," instructions to ignore prior context, unexpected references to other tool names). This is a genuinely new test category for most QA teams, closer to how a web application team tests for stored XSS than to anything in a typical API test suite, because the payload isn't trying to break your server — it's trying to reach the model through your server.

Rate limiting deserves a look beyond simple throughput protection. The specification requires servers to "rate limit tool invocations," and the obvious purpose is protecting your backend from a runaway agent loop. Test for that, but also test the case where a single agent session, driven by non-deterministic tool selection, calls the same tool many times in quick succession as part of legitimate-looking retry or exploration behavior — a pattern a human-driven client would rarely produce but a model iterating on a task can produce routinely. A rate limit tuned for expected human-client traffic patterns can either throttle a legitimate agentic workflow prematurely or fail to catch a genuinely runaway one, and the only way to know which is to test against realistic agentic call patterns rather than assuming REST-era traffic shapes.

Versioning your own tools needs a policy you actually enforce, not just a good intention. Given that the protocol provides no required per-tool version field, the discipline has to be self-imposed: decide, in writing, what counts as a breaking change to one of your tools (a renamed parameter, a changed default, a narrowed definition of an existing field, a changed error condition) versus a non-breaking one, and build a test that fails your own release pipeline when a breaking change ships without a corresponding major-version bump in your tool's name or a documented deprecation window. A pattern worth adopting from API practice that transfers cleanly here: expose tool_name_v2 alongside tool_name for a transition period rather than mutating tool_name in place, specifically because nothing downstream is guaranteed to notice the mutation happened.

Structured errors matter more than they do in most REST contexts, because the consumer is trying to self-correct. The specification distinguishes protocol errors from tool execution errors precisely because "clients SHOULD provide tool execution errors to language models to enable self-correction" — meaning a well-written error message is not just an operational nicety, it's the mechanism by which a model recovers from a bad call and tries again correctly. Test that your error messages for common failure conditions (invalid date, out-of-range value, missing permission) are specific enough for a model to act on, not just technically correct. A 400 Bad Request with no detail is a dead end for a model in exactly the way it's merely inconvenient for a human developer reading a log.

Metrics That Actually Tell You Whether This Is Working

Most teams that have connected agents to MCP servers are watching uptime and latency dashboards inherited from ordinary API monitoring, and those metrics answer a real question — is the server responding — without answering the question that actually matters for the failure modes described in this article: is the agent using these tools correctly. The table below distinguishes metrics worth building against metrics that create a false sense of coverage.

The distinction that matters most: the first two rows are what most existing monitoring already captures, and they are the metrics most likely to look healthy right up until one of the five failure modes causes real damage. The remaining four rows require deliberate instrumentation most teams haven't built yet, and they are the ones that would have caught the hypothetical scenarios described earlier in this article before a customer noticed.

An MCP Trust and Test Maturity Model

Given five structurally different failure modes, a useful governance response can't be a single checklist bolted onto existing API review. What follows is a four-level maturity model built specifically around the MCP failure surface described above, meant to help a team locate itself honestly and identify the next concrete investment, rather than treat "MCP governance" as a single yes/no gate.

Most organizations that have connected more than a handful of MCP servers in the last year are sitting at Level 0 or Level 1, largely because Level 1 is what "do what you'd do for a new API integration" naturally produces, and getting further requires recognizing that the failure modes are genuinely different, not just an extension of API governance.

A Risk Matrix for Prioritizing What to Fix First

Not every connected MCP server carries equal risk, and a governance program that treats an internal read-only documentation server the same as a third-party server with write access to production billing data will waste effort. The following illustrative risk matrix — a framework for scoring, not a source of real incident data — gives a way to prioritize.

Score each connected server against these six factors and prioritize the Level-3 testing investment (adversarial description review, output scanning, blast-radius review) on whatever lands in the high-risk column for two or more factors — in practice, this is almost always the third-party servers with write access and unmanaged update cadence, not the internal read-only ones engineers tend to worry about first because they built them.

Concrete Testing Techniques for Each Failure Mode

Maturity models and risk matrices set direction; they don't tell an engineer what to actually write on a Tuesday. Here is a technique per failure mode, with enough specificity to implement.

Schema and description snapshot testing (addresses Modes 1 and 2)

Treat every connected MCP server's tools/list response as a versioned artifact, exactly the way an OpenAPI spec would be treated for a REST dependency. Fetch it, hash or diff it against the last approved version, and fail a build (or at minimum, page a human) on any change — new tool, removed tool, changed description text, changed input schema, changed required fields.

# Pseudocode: MCP tool-contract snapshot diff, run in CI or on a schedule
import json, hashlib

def fetch_tool_list(mcp_client, server_name):
    response = mcp_client.call("tools/list", server=server_name)
    return response["tools"]

def normalize(tools):
    # Sort so ordering changes don't trigger false positives
    return sorted(tools, key=lambda t: t["name"])

def snapshot_hash(tools):
    canonical = json.dumps(normalize(tools), sort_keys=True)
    return hashlib.sha256(canonical.encode()).hexdigest()

def check_for_drift(server_name, mcp_client, approved_snapshots):
    current_tools = fetch_tool_list(mcp_client, server_name)
    current_hash = snapshot_hash(current_tools)
    approved_hash = approved_snapshots.get(server_name)

    if approved_hash is None:
        raise ValueError(f"No approved baseline for {server_name}; run approval flow first")

    if current_hash != approved_hash:
        diff = describe_diff(approved_snapshots.get(f"{server_name}_tools", []), current_tools)
        alert_and_block_deploy(server_name, diff)
        return False
    return True

This alone catches most of Failure Mode Two (drift) and forces a human review checkpoint for Failure Mode One (a changed description gets a second look before it reaches production, rather than reaching the model silently). It does not require the server operator's cooperation — it only requires the client to call tools/list and store the result, which any MCP client can do today.

Adversarial description review (addresses Mode 1)

Before approving a new or changed tool description, run it through a structured review — ideally both a manual read and an automated pass — looking specifically for: instructions directed at "the assistant" or "the model" rather than describing behavior to a human reader; conditional instructions ("if the user asks about X, also call Y"); urgency or compliance framing ("REQUIRED," "MUST," references to audits or regulations that have no plausible connection to the tool's actual function); and instructions about other tools embedded inside one tool's description (the tool-shadowing pattern). A simple automated heuristic — flagging descriptions containing second-person imperative language directed at an AI system, or containing references to unrelated tool names — catches a meaningful share of the crude cases and is cheap to run on every tools/list diff.

Output-side injection scanning (addresses Mode 3)

Because this risk lives in data flowing through a tool, not in the tool's definition, it needs a runtime check, not a one-time review. Practically, this means treating tool results the way a security-conscious team already treats user input: scan text content returned by tools for embedded instruction-like patterns before it's added to the model's context, and — more robustly — architect the highest-risk tool chains (particularly "read external content, then take an action with side effects") so that any action step requires either explicit user confirmation or passes through a separate, more constrained model call that only sees the retrieved content and a narrow classification task, rather than the full agentic context with tool-calling ability enabled.

Sampled non-determinism testing (addresses Mode 5)

Instead of running an integration test once, run each meaningful test case N times (20–50 is a reasonable starting point for a moderately complex agent) and record the distribution of tool-call sequences produced, not just whether the final answer looked right. Flag a regression when the distribution shifts meaningfully — a tool that used to be called in 95% of runs now appears in 60%, or a confirmation step that used to always precede a write action starts getting skipped in a nontrivial fraction of runs.

# Pseudocode: sampling-based regression detection for agent tool-call behavior
from collections import Counter

def run_agent_n_times(agent, prompt, n=30):
    call_sequences = []
    for _ in range(n):
        trace = agent.run(prompt)
        sequence = tuple(call.tool_name for call in trace.tool_calls)
        call_sequences.append(sequence)
    return call_sequences

def summarize_distribution(sequences):
    return Counter(sequences)

def flag_regression(baseline_dist, current_dist, threshold=0.15):
    baseline_total = sum(baseline_dist.values())
    current_total = sum(current_dist.values())
    all_sequences = set(baseline_dist) | set(current_dist)
    regressions = []
    for seq in all_sequences:
        baseline_pct = baseline_dist.get(seq, 0) / baseline_total
        current_pct = current_dist.get(seq, 0) / current_total
        if abs(baseline_pct - current_pct) > threshold:
            regressions.append((seq, baseline_pct, current_pct))
    return regressions

This is a materially different mindset from most existing QA tooling, and it is the piece of this framework most teams are missing entirely, because almost no off-the-shelf test framework treats "run it 30 times and look at the spread" as a default practice.

Blast-radius auth review (addresses Mode 4)

Rather than reviewing each server's auth in isolation, map every credential an agent session can present, list every downstream system reachable through that set of credentials, and ask: if this specific credential were leaked today, what is the maximum reachable damage, and is that damage smaller than what a single compromised server would already have implied? Where the answer is "no, one leaked token reaches far more than any single server justifies," that's a scope-minimization finding, directly addressable using the pattern the specification itself recommends: minimal initial scopes, with incremental step-up authorization requested only when a higher-privilege operation is actually attempted.

A Pre-Connection Vetting Checklist

Use this before approving any new MCP server for use by a production agent — internal or third-party, low-risk or high-risk. It is deliberately not exhaustive of general security review; it's scoped to the MCP-specific failure modes above.

1. Provenance. Who maintains this server, and what is their update and disclosure process? Is there a changelog, and does it distinguish breaking from non-breaking changes?
2. Baseline snapshot. Capture and store the full tools/list response before first use, including every description and input schema, as the approved baseline for future diffing.
3. Description review. Read every tool description as if it were untrusted input, specifically checking for instructions directed at the model, conditional cross-tool instructions, and unrelated compliance/urgency framing.
4. Scope audit. What is the minimum OAuth scope or credential this server actually needs for the use cases it will serve? Is that narrower than what it's being granted?
5. Token handling check. Does this server (if acting as a proxy to another API) validate that tokens were issued specifically for it, or does it forward client-presented tokens downstream unmodified? (If unmodified: this is a token-passthrough anti-pattern the specification forbids, and a hard no for anything touching sensitive systems.)
6. Output content exposure. Does this tool read content originating outside your organization's control (user submissions, external web content, third-party message content)? If yes, its output must be routed through injection-aware handling before being trusted as agent context.
7. Action reversibility. Does this tool take irreversible actions (send, delete, transfer, execute)? If yes, require an explicit confirmation step regardless of what the model decides is warranted.
8. Blast-radius mapping. What other systems become reachable if this server's credential, or the agent session using it, is compromised?
9. listChanged wiring. Does your MCP client actually act on notifications/tools/list_changed by re-fetching and re-diffing, or does it cache the tool list for the life of a session and ignore updates?
10. Non-determinism baseline. For any workflow involving this tool that has real-world side effects, has the tool-call distribution across repeated runs been sampled and recorded as a baseline before go-live?
11. Ownership assignment. Is there a named owner responsible for reviewing this server's next update, or does responsibility default to whoever happened to add the connection?
12. Removal plan. If this server needs to be revoked quickly (compromise, vendor issue, deprecated use case), is there a documented, tested way to disconnect it without taking down the entire agent?

Who Actually Owns This

The failure modes above span security, platform engineering, and QA in a way that, in most organizations, currently falls into the gap between all three — security teams often haven't been briefed that MCP servers exist in the stack at all; platform teams are focused on getting integrations working, not on adversarial review of description text; and QA teams, when they're looped in, default to the API-testing muscle memory that this article has argued doesn't transfer cleanly.

The specific assignment matters less than making sure every row above has a name attached to it. In every organization examined for this piece — including the hypothetical support-agent scenario at the top — the actual failure was not that no team was competent to catch the problem; it was that no team had been told this was now part of their job.

A useful diagnostic for whether this gap exists in a given organization today: ask three separate people — a security lead, a QA lead, and whoever owns the agent or AI platform — to independently produce the full list of MCP servers currently connected to any production system. If the three lists don't match, or if any of the three can't produce a list at all, the ownership gap described above is already active, regardless of how mature any individual team's own testing practices are. This is a cheap exercise to run and a reliable signal, because it tests for the actual failure condition — a decision or observation that nobody was assigned to make — rather than testing for the competence of any one team, which is rarely the real bottleneck.

Startups, Scale-Ups, and Enterprises Face This Differently

A five-person startup wiring an agent to three MCP servers doesn't need a four-level maturity model or a dedicated blast-radius review process — it needs the pre-connection checklist above applied honestly, and a habit of treating every new server the way a careful engineer already treats a new production dependency. The realistic risk at this stage is mostly Failure Modes One and Two, because the number of connected servers is small enough that a founding engineer can still personally read every tool description, and the cost of getting caught by drift is annoying rather than catastrophic.

A scale-up connecting a growing agent surface to a dozen or more servers across multiple teams is exactly where Level 0 and Level 1 governance stops being sufficient, because no single person can hold the full picture of what's connected in their head anymore, and that's usually the point at which the first serious incident happens — not because the team got worse at engineering, but because the informal review process that worked at five servers silently stopped working at fifteen. This is the stage where the snapshot-diffing and inventory work described above stops being optional.

An enterprise operating MCP connections across many business units, often with regulatory exposure (healthcare, financial services, anything handling PII at scale), needs the full blast-radius and auth-review discipline from day one, because the cost of a single high-risk server having under-scoped access compounds across every downstream system that credential can reach — and because regulators and auditors increasingly expect a documented answer to "what can this AI system actually touch," which none of the failure modes above make easy to answer without the inventory and scope-mapping work already done.

Enterprises also face a coordination problem the other two categories mostly avoid: multiple business units frequently connect overlapping MCP servers independently, each unaware of the other's approval decision, credential scope, or review outcome. A finance division and a customer-support division might both connect the same third-party ticketing server, one having reviewed its tool descriptions carefully and scoped its credential narrowly, the other having accepted the vendor's default configuration without the same scrutiny — and because nothing in the protocol requires centralized registration, the weaker of the two configurations is often the one that determines the organization's actual exposure, since an attacker only needs to find the least-scrutinized path in. This is less a technical limitation of MCP than a predictable consequence of any protocol designed to let individual teams self-serve integrations without a mandatory central checkpoint, and it argues for the server inventory described earlier being genuinely centralized — visible across business units, not maintained separately by each one — well before an enterprise reaches the scale where reconciling divergent configurations after the fact becomes its own project.

Frequently Asked Questions

Is MCP itself insecure? No — and it's worth being precise here. The protocol's own specification devotes explicit, normative language to security and trust considerations, requires human-in-the-loop confirmation for tool invocations, and treats tool descriptions and annotations as untrusted by default unless proven otherwise. The disclosed vulnerabilities discussed in this article (CVE-2025-54136, CVE-2025-54135) were implementation flaws in a specific client, not protocol design flaws, and the protocol's maintainers document the confused-deputy and token-passthrough risks explicitly precisely so implementers don't repeat them. The risk isn't that MCP is unsafe by design; it's that the protocol intentionally leaves enforcement to implementers, and most implementers — including internal teams building their first MCP server — haven't yet built that enforcement.

Can we just require every MCP server to be reviewed once before connecting it? That addresses Failure Mode One at a point in time, but not Failure Mode Two — a server reviewed and approved today can change its tool definitions next week with no compatible-versioning guarantee forcing a re-review. A one-time gate needs to be paired with the ongoing snapshot-diffing described above, or the review becomes stale the first time the server updates.

Does using only first-party or well-known vendor MCP servers eliminate this risk? It substantially reduces Failure Mode One risk from deliberate malice, but does not remove Failure Modes Two, Three, Four, or Five, all of which can occur with an entirely well-intentioned, reputable server. A reputable vendor can still ship a breaking schema change, still expose a tool that reads external content susceptible to injection, still be granted broader scope than necessary, and its tool will still be selected non-deterministically by the model regardless of how trustworthy the server's author is.

How is this different from testing a chatbot's conversational quality? Conversational quality testing evaluates whether the model's language output is appropriate, helpful, and on-brand. This article is about whether the model's actions — the tools it invokes, with what arguments, based on what it was told those tools do — are correct, safe, and consistent, which is a systems-integration and security question, not a language-quality one. A model can produce a perfectly fluent, friendly response while having just exfiltrated data through a poisoned tool description.

What's the single highest-leverage first step for a team starting from nothing? Build the inventory: a list of every MCP server currently connected to any production agent, who owns it, what it can access, and what credential it uses. Most organizations examined in the course of researching this piece did not have this list before being asked for it, and every other technique in this article depends on that list existing first.

Does rate-limiting or sandboxing tool execution solve the non-determinism problem? No — rate limiting and sandboxing address blast radius and execution safety, which is valuable, but they don't address the underlying issue that the same input can lead a model to select different tools or arguments across runs. That requires the sampling-based testing approach described above; it's a measurement problem, not a containment problem.

Should QA own MCP testing, or does this belong entirely to security? Neither exclusively, and treating it as one team's job is part of why the gap described in this article exists. Security has the threat-modeling instinct to catch adversarial tool descriptions and auth blast-radius issues; QA has the testing-infrastructure instinct to build the sampling and drift-detection tooling; platform engineering has the access and deployment authority to wire either team's findings into an actual gate. The responsibility map earlier in this article is deliberately split across all three because none of them currently owns the full picture on its own in most organizations.

We only use MCP servers built by major AI vendors — does that change the calculus? It changes which failure modes are most likely, not whether the discipline is needed. A major vendor's server is less likely to ship a deliberately poisoned description, but vendor servers still update on their own release schedule, still can expose tools that read external content, and still get connected with broader scope than the specific use case requires because broad scope is more convenient to grant once than to revisit later. The pre-connection checklist and drift-monitoring apply regardless of who wrote the server; only the adversarial-intent assumption changes.

The Governing Principle

Every failure mode described here traces back to one structural fact: MCP moved the integration contract from a place machines enforce to a place a probabilistic model interprets, and most organizations kept using the testing habits built for the first kind of contract on the second kind. That's not a reason to slow down MCP adoption — the standardization the protocol provides is real and worth having. It's a reason to stop asking "did the integration work in the demo" and start asking a more specific question of every connected server: if this tool's description, schema, or output were subtly wrong, adversarial, or simply different next week, what is the first signal that would tell us — and is that signal something a person will actually see, or something that will just quietly change what the agent does? If the honest answer today is "we'd find out from a customer complaint," the inventory and snapshot work described in this article is the place to start, not the risk matrix or the maturity model — those only matter once there's something concrete to point them at.

Where QAtronic can help: teams that have moved quickly on agent integrations often reach this point with a working system and no test coverage for exactly the failure modes described above. QAtronic builds the contract-snapshot pipelines, adversarial description review processes, and non-determinism sampling harnesses this article describes into existing CI and agent platforms, treating MCP-connected tools as a distinct testing discipline rather than an extension of API test suites that were never built for a probabilistic caller.