Conformance checks
These pages describe checker version VeronikAI/0.1+attestations.1 (signing.py). A2A card checks use suite veronikai-a2a-get-card version 1. MCP checks use suite veronikai-mcp-conformance version 2. If the code changes, the version string on a signed result should change too.
What is checked
Uptime asks "is anyone home?" Conformance asks a different question: does the public paperwork follow the protocol's rules?
A shop can be open and still have a blank or broken sign. That shop would be up and fail conformance.
We only inspect structure. We never follow instructions hidden inside a card or server.json. Inputs are treated as untrusted data (validate.py).
How it works
A2A agent card — suite veronikai-a2a-get-card version 1
After a successful GET of the card (probe.py → validate.a2a_card):
The JSON object should include:
- non-empty name, description, and version;
- an https:// url, or a v1.0-style supportedInterfaces / additionalInterfaces list whose items have URLs;
- a capabilities object;
- non-empty defaultInputModes and defaultOutputModes;
- at least one skill, and every skill must have id, name, description, and a tags list.
A card that is not valid JSON fails. A card that is not public (HTTP 401/403) fails with "agent card not public". HTTP 402 is not judged (conformance unknown); we do not pay.
MCP — suite veronikai-mcp-conformance version 2
The official MCP Registry already rejects many schema-invalid server.json files, so schema-only checks would pass almost everyone. Version 2 adds quality checks plus one live GET.
Quality (validate.mcp_quality) fails when:
- there is no description, the description is under 20 characters, it is a placeholder ("hello world", "an MCP server", …), or it only repeats the name;
- the version is missing or is not semver (x.y.z);
- there are no remotes, a remote is not streamable-http or sse, the remote URL is not https, or it points at a placeholder or private host (example.com, localhost, private IPs);
- a listed repository URL is invalid.
Warnings (they do not fail the server) include: no repository or website URL, version 0.0.0, description equal to the title, a templated URL that still needs user values, or a probed endpoint that differs from the declared remote.
Transport (validate.mcp_transport) judges that one GET (Accept: text/event-stream):
- Streamable HTTP should return an SSE stream (text/event-stream) or HTTP 405.
- Legacy SSE should open with an endpoint event.
- HTTP 401 should carry WWW-Authenticate: Bearer or publish RFC 9728 metadata. Missing metadata on 401/403 is a warning, not a hard fail, if Bearer or metadata is present as required.
- An HTML page, a 404, or a 5xx fails.
- HTTP 402 or 429 is unknown (paywall or rate limit).
- A sessionless JSON-RPC error instead of 405 is a warning.
MCP listings that have no server.json (they were not in the official registry) get conformance n/a. We still record uptime.
What pass and fail mean
| Result | Meaning |
|---|---|
| pass | No hard failures. Warnings may still be listed, prefixed with warn:. |
| fail | At least one hard rule broke. The reasons list says which. |
| unknown | We could not finish the check fairly (no answer, 402, 429, URL template, time cap). |
| n/a | There was nothing of that protocol's paperwork to judge. |
An agent can be up + fail (home, paperwork wrong) or down + fail (for example A2A with no card). Pass never means "this agent is good at its job" or "its tools work". We did not call a tool.
Limits
- We do not keep the fetched A2A card body after judging it, so skill names inside the live card are not stored for search today.
- Descriptions in our agent table are stored from the listing (up to 500 characters), not re-copied from a freshly fetched card.
- We do not send MCP tools/list, so we do not verify the server's real tool inventory.
- Quality rules can be strict: a short but honest description under 20 characters fails.
Checker version
VeronikAI/0.1+attestations.1, with the suite id and suite version named on each signed record.