THE EXECUTION LAYER
Comparability and standings
The one rule
A time belongs to the point it was measured at and the machine it ran on. Fix both and two numbers compare. Release the point and the reading is about scaling; release the machine and it is about portability. Neither of those is a ranking, and no page averages across either.
Everything below is this rule applied.
The current answer
Per (implementation, machine, point) the current answer is the newest
run's fold of that point. Older runs are kept — they are history, not a
disagreement — but one cell shows one number.
Folds are computed on the way out and never stored: min, median,
mean, max over the seed timings, the same over memory, and the
verifier's metrics folded as ranges. The median is what tables quote;
the spread is shown beside it, not averaged away.
What may be ranked
Standings exist per specification, at one point on one machine. An implementation is ranked there when its current answer:
- has a timing;
- lost no cases — a seed that produced no answer disqualifies;
- is judged and passed under the bundle's bar: every case passed when no bar is declared, or the declared proportion of at least the minimal sample.
Everything else is listed below the table as measured, not ranked, with the reason said plainly: seeds failed, seeds lost, or the run was below the specification's sample and there is no verdict to rank.
A fast wrong answer is not a contender, and a run too small to judge is not an answer yet.
The compare views
One page holds four questions, each honest about what it fixes:
| view | fixes | releases | reads as |
|---|---|---|---|
| like for like | point + machine | — | a comparison |
| how it scales | machine | point | scaling behaviour |
| where it runs | point | machine | portability |
| by library | point + machine | — | best answer per FHE library |
Cells carry more than the clock: the verifier's accuracy, the declared security level with its provenance, ring and capacity, key counts, memory — so two answers are compared on their conditions, not just their speed.
Classes
Requirements set the class an implementation competes in. A GPU answer and a bare-CPU answer produce incomparable numbers, and one table holding both would rank hardware invisibly. Declaring more than the code uses is self-defeating: ask for a GPU and you are compared with those who use one.
Mechanisms compare the same way: an FHE time next to a plaintext time is a fact about the cost of cryptography — the pages will show it, labelled — never a ranking between the two.
The reference machine
The catalogue quotes one machine's times — the reference runner — so that a number seen on a kernel's page always means the same conditions. Other machines' runs confirm correctness and document portability; their times compare only with their own.