How to Debug MCP Connectors Across ChatGPT, Claude, Grok, and More
This morning we were debugging an OpZero-hosted MCP server that completed OAuth successfully with ChatGPT and still failed to connect.
That investigation uncovered several interoperability issues across ChatGPT, Grok, and the official MCP Inspector. Some were protocol-adjacent rather than protocol violations: session IDs that are optional in the transport but expected by a client, authorization metadata that is valid but arranged in a way another client will not follow, and declared capabilities that our own tests accepted but the reference client immediately rejected.
The useful outcome was not the individual fixes. It was a better way to test MCP servers.
We extracted that work into a public project:
The problem with “MCP compliant”
There is a meaningful difference between satisfying the MCP specification and working reliably across MCP clients.
A server can be legal according to the transport specification and still fail a particular client. A schema can be valid JSON Schema and still be interpreted poorly by an SDK. OAuth discovery can be standards-compliant while relying on redirects or issuer relationships that a client does not support.
At the same time, building a test suite entirely around observed client behavior is dangerous. Client quirks can easily become accidental requirements, especially when nobody remembers where a rule came from six months later.
So the harness treats conformance as two separate questions:
- What do the protocol and specific clients appear to require?
- Can an independent reference implementation actually use the server?
Both need to pass.
Gate 1: a client compatibility matrix
The first gate is an executable registry of MCP client requirements.
Profiles currently cover surfaces including Claude connectors, Claude Code, ChatGPT, the OpenAI Responses API, Grok, xAI, IDE clients, the MCP specification floor, and an editable platform-policy profile.
Each requirement maps to an executable check.
The important part is that requirements also carry evidence and confidence.
A requirement can be based on:
- Observed behavior — we watched a real client do it.
- Documented behavior — the specification or vendor documentation says it.
- Inferred behavior — we have reason to believe it is required, but have not verified it directly.
Inferred behavior can be reported, but it cannot fail CI.
That rule prevents a compatibility hypothesis from quietly turning into permanent platform policy.
The matrix also requires explicit treatment of skipped checks and known gaps. If a surface cannot support a check, it declares why. If a compatibility problem is knowingly accepted, the expected gap is recorded exactly.
When the gap is later fixed, CI fails until the exception is removed.
Gate 2: the official MCP Inspector
The second gate runs the official MCP Inspector CLI against the server over real loopback HTTP.
This exists because the first gate still has a fundamental limitation: it is our interpretation of the evidence, executed by our own probe.
We learned that immediately.
Our internal conformance suite was green, but the Inspector failed before its first tools/list.
The server advertised the MCP logging capability but did not implement logging/setLevel. The Inspector correctly called the method after initialization, received -32601, and abandoned the session.
Our own checks had never asked the question.
The Inspector also surfaced schema portability issues and incomplete schema conversion behavior that were difficult to see from isolated protocol tests.
Neither gate is sufficient by itself.
The client matrix gives broad provider coverage but can encode our mistakes.
The Inspector provides an independent implementation but represents one client.
Together they provide substantially better confidence.
Testing the tests
The repository also includes a small known-good MCP server with switchable defects.
Individual behaviors can be deliberately broken, such as omitting a session ID or publishing inconsistent authorization metadata. Mutation tests then assert that the corresponding conformance check turns red.
This is an important property for any conformance suite.
A test that only runs against working implementations has not demonstrated that it can detect the failure it claims to detect.
The fixture provides calibration for both the checks and future contributions.
What came out of the debugging work
Several of the checks in the public registry came directly from real interoperability failures we hit while operating OpZero.
For example, the harness now captures cases around:
Mcp-Session-Idbehavior during initialization- authorization-server metadata consistency
- same-origin authorization routing
- RFC 9207 issuer signaling
- Client ID Metadata Documents
- resource-bound OAuth flows
- declared MCP capabilities that are not actually implemented
- schema portability across clients
These are useful precisely because they are no longer just debugging notes.
They are executable.
Why publish it
OpZero needed this because we deploy MCP servers for other people. A compatibility issue in our runtime becomes a compatibility issue for every server built on top of it.
But none of the underlying problems are unique to OpZero.
Anyone implementing an MCP server has to navigate the same boundary between specification correctness, SDK behavior, OAuth standards, and production client expectations.
There is little value in every team independently discovering the same edge cases and turning them into private regression tests.
Our preference is to publish the evidence and the tests as we find them.
If a profile is wrong, it should be corrected with better evidence. If a new client behaves differently, that behavior should become a reproducible check. If the protocol evolves and makes an old workaround unnecessary, the compatibility matrix should reflect that too.
The goal is not to create another MCP specification.
It is to make real-world MCP interoperability easier to measure.
The project is here: