Skip to content
CAI
Software that uses CAICheck a score

Rubric versions

A versioned, contestable rubric.

The rubric is the set of rules that turns measurements into a CAI score. A score can only be compared with another score if the rules behind both of them held still, so the rubric is versioned: every score names the version it was worked out under, and that version does not change afterwards. Where a project needs it, one version can be fixed for the length of an agreement, so any difference between the first score and the last comes from the code.

The three guarantees

How the rules change, and what holds still when they do.

Versioned

Any change that could move a score for code that has not itself changed becomes a new version of the rules. Every survey names the version it was scored under, so when a number moves the record shows which of three things happened: the code changed, the score was taken under a newer version, or newly disclosed vulnerability data moved a security finding under rules that stayed the same.

Published

Every rule that can move a score is published before it is used. Anyone can work the number out again from rules they can read.

Freeze / pin for a contract

A project can be tied to one fixed version of the rules for the length of an agreement. The score agreed at the outset and the score at completion are then measuring the same thing, and neither side can be surprised by a rule that arrived somewhere in between.

Disputes, from the standard's side

A dispute improves the measuring. It does not move the number.

Contest a finding

A finding can be contested.

A scored finding can be disputed, and the dispute goes to whoever issued the score. Where the finding turns out to be wrong, the fix goes where the fault was: a detector that misread the code is the implementation's to fix, and a rule that gave the wrong answer is the standard's. The score itself is not adjusted: a dispute is meant to improve what gets measured, not to open a route to a better number.

Advisory data

Advisory data still refreshes.

A fixed rubric version stops the rules moving. New information can still arrive: a vulnerability disclosed publicly for the first time can change a security finding under a rubric version that has not changed at all. That is recorded in the survey's changelog, so the two can be told apart.

Read the rules before you are scored against them.

Software that uses CAI → /implementations · Who decides what the rules say → /governance