Reference scorer / command line
The open-source reference scorer.
The arithmetic that turns evidence into a CAI score is published, and it is also a program you can run. Give the reference scorer the evidence from a survey and it produces the same number between 0 and 100 that the survey published, offline, on your own machine.
Source: github.com/code-assurance-initiative/CodeAssuranceIndex. Packages: Cai.Scoring and Cai.Delivery, published by the Initiative to GitHub Packages. An anonymous nuget.org feed is not yet available — until it is, build the scorer from source.
What it guarantees
Public arithmetic, runnable anywhere.
Open source
The scoring arithmetic is published in full, and the reference implementation of it lives at github.com/code-assurance-initiative/CodeAssuranceIndex. Every step from evidence to headline is there to be read, run, and rewritten by anyone who would rather use their own.
Offline-runnable
Score an evidence package on your own machine, with no service to call, no account to make and no network connection at all. The package names the rubric version it was scored under, so the scorer knows which rules to apply and reaches the same result every time.
The guarantee
Run the open scorer over a package's evidence, pinned to the rubric version that package names, and you get the number the survey published. Anyone can run it, and everyone who does gets the same result.
REFERENCE
Score evidence locally
- In: a CAI evidence package, as JSON.
- Out: the same 0 to 100 CAI, with the score for each of the ten lenses.
cai score ./evidence.json --rubric <version>The packages are Cai.Scoring and Cai.Delivery, published by the Code Assurance Initiative. They are on GitHub Packages, which requires a GitHub account even to read — so for now the honest instruction for an anonymous third party is to clone the repository and build it. Every evidence package names the rubric version it was scored under, so the scorer never has to guess which rules you meant.
The scorer works on evidence somebody else produced.
The reference scorer recomputes the score from the evidence in a package. The detectors that produced that evidence are not part of the open distribution, and each implementation builds its own. That is the boundary: the standard publishes everything needed to check the arithmetic, and the measuring stays with whoever did it.
Run the arithmetic yourself.
How the number is worked out → the standard · What gets measured → dimensions and lenses · How the rules change → rubric versions