Audit
Certified leaderboard
Send a contamination certificate, and the server checks it before a row exists. Rankings appear only once a published quorum of comparable certified runs exists, so there is no board yet.
PLANNED
The machinery is live: three /v1/registry routes accept,
verify and count submissions today. What does not exist is the public
comparative view, and the rule that has to be met before one is built
is published below.
- REST
- /v1/registry
- any paid plan
- private by default
The artifact
A submission, and the cell it lands in
Two real requests against a local server on 2026-08-27. The first sends the unsigned golden certificate from this repository; the registry refuses it, because the signing key's fingerprint is what quorum counts and an unsigned file has none.
{
"code": "certificate_unverifiable",
"message": "This certificate did not verify, so it was not stored: the certificate carries no signature block, so there is no key to register it under. Run `certverify verify` against it before sending it again.",
"param": "unsigned"
}
The second sends a signed certificate over a real harness run, listed rather than private. It is accepted, and the cell it belongs to now reports its counts against the published rule.
[
{
"window": "2023-01-03..2023-01-13",
"arm_set": "main+redacted+shifted",
"listed": 1,
"fingerprints": 1,
"quorum": false,
"needs": {
"listed": 5,
"fingerprints": 3
}
}
]
needs is the published rule, repeated on every row, so
the counts can be read against it without leaving the response. This
cell holds one listed submission and one fingerprint, against a bar
of five and three.
The rule
Quorum, stated once
A cell reaches quorum when it holds at least five listed submissions signed by at least three distinct keys.
A cell is one pair: the run window the agents were scored over, and
the set of arms they were run under.
GET /v1/registry/cells reports the two counts and the
boolean for every cell that has at least one listed submission.
The registry page has the reasoning behind
both halves of the bar and what a listing does and does not stand
behind.
How it works
What the server checks before a row exists
Three of the five checks, over the bytes you sent
The server runs body_digest,
receipt_body and signature, with the same
rules certverify uses. The
other two, trajectory and
bundle_manifest, compare a certificate against files
you hold on your own machine, and a server holds neither. A listing
does not stand behind those two.
There is no published key file for a key we have never seen, so the signature is checked against the copy the certificate carries and that key's fingerprint is recorded. A pass tells you the file is internally consistent and that whoever holds the matching private key signed it. Which person or company that is stays an open question here.
Why a refusal is reproducible before you send
error.param | What happened | certverify exit |
|---|---|---|
malformed | the file will not parse, or is not version 1 | 2 |
body_digest | body_sha256 does not cover the bytes the file ships | 3 |
receipt_body | the embedded receipt is not the one the binding names | 3 |
signature | the signature does not verify, or the key is unusable | 4 |
unsigned | the certificate carries no signature block | — |
subject | the certificate names no run window, no arms, or no model | — |
The last two are the registry's own rules.
certverify reports an unsigned certificate's signature
check as skipped and still calls the file complete; here the
fingerprint is what quorum counts, so a certificate without one has
nowhere to go.
One certificate is one row
A second account sending the same certificate gets a 409. A
certificate is a public file, so anyone holding it could send it,
and counting it once per sender would let a cell reach quorum on
one run forwarded five times. A run the harness marked invalid is
accepted: the certificate is genuine and what it certifies is a run
that was thrown out, so the row records valid: false
and the numbers that go with it.
{
"cert_sha256": "sha256:7fa3b39f16b9478bff8e410b86882ca0eee1b27826ecbf208e5477846796d835",
"pubkey_fingerprint": "4edfcdf419846c26",
"window": "2023-01-03..2023-01-13",
"arm_set": "main+redacted+shifted",
"subject_model": "pit-harness tests/stub-agent@0.1.0",
"metrics": {
"valid": true,
"reference_arm": "main",
"largest_measured_delta": 0,
"positive_control": "mask_holds",
"reasoning_audit": "not_run"
},
"submitted_at": "2026-08-27T18:55:05Z",
"submitter_key_id": "key_84887c96e987ca3712993c8a073a7a6a",
"visibility": "listed"
}
Integration
Submitting
$ curl -sS -X POST \
'https://api.pit.aqx.llc/v1/registry/certificates?visibility=listed' \
-H "Authorization: Bearer $PIT_KEY" \
-H 'Content-Type: application/json' \
--data-binary @certificate.json
Send it with --data-binary rather than -d:
-d strips newlines and the digests are taken over the
bytes that arrive. The file is capped at 1 MiB, which leaves room for
a run about forty-five times the size of the 23,347-byte golden. Any
paid plan may submit, and the submission has to come from an API key
rather than a browser session, because the key id is recorded on the
row.
Every submission is private by default. A private submission appears
in your own list and in no count, no aggregate and no page anyone
else can reach. Sending ?visibility=listed adds one to
its cell's listed count and one fingerprint; its metrics
are stored and served to nobody but you. Send the same certificate
again with ?visibility=private to take it back off.
Related