Skip to content
CAI
Software that uses CAICheck a score

ahamez/protox

79.0

Strong · 18 September 2026

5.8k

lines of production code

Elixir

primary language

1

measurement over time

CAI band scale
CAI lens gauges

What this system is

Protox is an Elixir library that provides high-performance Protocol Buffers encoding and decoding through a rewrite that shifts code generation to compile time. It includes a CLI task for generating Elixir modules from .proto files and supports Google's well-known types. The system also features a comprehensive conformance test runner to validate standard compliance and a benchmarking harness for performance analysis.

How it got here

2016 — Protox v2 rewrite and tooling standardization

7 changes.

The project underwent a major architectural shift by rewriting the code generation engine for Protox v2, moving from runtime-based generation to a compile-time architecture with strict version checking. This core update was accompanied by a comprehensive overhaul of the development environment, including the adoption of Elixir 1.15, new linting and formatting standards, and expanded test coverage to ensure correctness and conformance.

2020–2021 — conformance testing and CLI generation

5 changes.

This period focused on enhancing the library's reliability and usability by introducing a comprehensive conformance test suite and a new command-line interface for code generation. Key developments included adding a Mix task and escript to automate protobuf validation, implementing the \mix protox.generate\ command, and defining schemas for Google's well-known types to ensure robust decoding and encoding.

2024–2025 — Benchmarking and testing infrastructure overhaul

6 changes.

This period focused on restructuring the benchmarking harness to support realistic, production-like payload generation and comprehensive schema coverage, including integration of Google and OpenTelemetry datasets. Concurrently, the project upgraded its test infrastructure by replacing :propcheck with :stream\_data for property-based testing and enforced commit signature verification via pre-push hooks.

Features

Add conformance test runner build and execution task

A new Mix task, \Mix.Tasks.Protox.Conformance\, has been added to the \conformance/mix\ location. This task automates the process of building the protobuf conformance test runner from source (using CMake with specific configuration flags) and executing it against the \protox\_conformance\ test suite. It supports options to control verbosity (quiet mode), force a rebuild of the runner, or perform a compile-only step, providing a streamlined way for users to verify protocol buffer conformance.

conformance/mix · high confidence

Add manual schema definitions for Google Protobuf well-known types

The library now includes explicit \MessageSchema\ definitions for Google Protobuf well-known types (such as \Any\, \Timestamp\, \Duration\, \FieldMask\, \Struct\, \Value\, and scalar wrapper types like \BoolValue\ and \Int32Value\) in \lib/google/protobuf\. These files define the structure of these standard types using the internal \Protox.Define\ DSL, enabling the library to recognize and process these specific message formats during decoding and encoding.

lib/google · high confidence

Expanded benchmark schema coverage with synthetic and vendor datasets

The benchmark harness now includes a broader set of Protocol Buffers schemas to ensure more comprehensive performance testing. This adds synthetic message definitions (edge, maps, and varying field counts from 5 to 200) to isolate specific encoding paths like NaN/Infinity handling, map operations, and deep nesting. It also vendors official benchmark datasets from Google (proto2 and proto3), OpenTelemetry trace protocols, and Prometheus remote-write schemas, providing realistic production-like message shapes for comparison against other implementations.

benchmark/protos · high confidence

New benchmark harness with configurable mix tasks and realistic payload generation

The benchmarking capability has been restructured into a dedicated mix task (\mix protox.benchmark.run\) that allows users to run encode/decode benchmarks with configurable durations, warmup, and profiling modes. The harness now uses a pre-generated, committed corpus of benchmark payloads (\benchmark\_payloads.bin\) to ensure consistent, comparable results across runs and machines. This corpus is populated using a new \Protox.Benchmark.RealisticValues\ module that generates production-like data distributions, replacing the previous edge-case-heavy generation. Users can also generate new payloads or aggregate results into HTML reports using the provided mix tasks.

benchmark · high confidence

New conformance test runner escript for protobuf validation

A new escript (\Protox.Conformance.Escript\) has been added to serve as the execution target for the protobuf conformance test suite. This tool reads conformance test requests from standard input, decodes them using the \Conformance.ConformanceRequest\ schema, and processes them by decoding the payload into Elixir structs and re-encoding them to verify round-trip integrity. It supports specific test message types (TestAllTypesProto2 and Proto3) and outputs results to a timestamped report file, handling serialization errors and unknown fields gracefully.

conformance/protox · high confidence

New mix protox.generate task for CLI code generation

Users can now generate Elixir code from \.proto\ files directly via the command line using the new \mix protox.generate\ task. This task accepts options to specify the output path (\--output-path\), include paths (\--include-path\), a module namespace (\--namespace\), and a flag to generate multiple files for parallel compilation (\--multiple-files\).

lib/mix · high confidence

Behavioural changes

Enforce commit signature verification on push

The pre-push hook now validates that all commits not yet pushed to the remote are cryptographically signed. If unsigned commits are detected, the push is blocked with an error message, ensuring that only signed history is shared.

.lefthook · high confidence

Expanded benchmark payload generation with real-world datasets

The benchmark harness now generates a broader set of test payloads, including specific inputs for OpenTelemetry traces and Prometheus write requests, alongside existing synthetic schemas. It also incorporates real captured payloads from Google's benchmark suite to ensure more realistic encoding and decoding measurements.

benchmark/mix/tasks/protox/benchmark/generate · high confidence

Protox v2 code generation with compile-time version mismatch detection

The library now generates code using generator version 2, introducing a compile-time check that raises an error if the runtime version of Protox does not match the version used to generate the protobuf modules. This ensures compatibility between the generated structs and the runtime library, preventing silent failures due to version drift.

lib · high confidence

Protox v2: Complete code generation rewrite with compile-time schema validation

The \lib/protox\ library has been rewritten for version 2, replacing the previous runtime-based code generation with a new compile-time architecture. The library now uses a \Protox.Define\ macro to generate Elixir structs, encoders, and decoders directly from \Protox.MessageSchema\ definitions at compile time, significantly improving performance and reducing runtime overhead. This change introduces a strict version check in generated modules to prevent incompatibility between the generated code and the runtime library. Additionally, the new implementation enforces Protobuf 3 field presence semantics, changes the unknown fields struct key to \\_\uf\\_\, and removes the deprecated \Protox.Encoder.encode/1\ API in favor of the new \encode/1\ and \encode!/1\ functions that return \{:ok, iodata(), size}\ tuples.

lib/protox · high confidence

Refactored benchmark harness with separate run and report tasks

The benchmarking workflow has been restructured into two distinct Mix tasks: \mix protx.benchmark.run\ and \mix protx.benchmark.report\. The run task now handles execution with configurable options for warmup, time, and profiling scopes (time, memory, calls), while the report task is responsible for generating console and HTML outputs from saved Benchee data. This separation allows users to run benchmarks with specific configurations and generate reports independently, improving flexibility and clarity in the benchmarking process.

benchmark/mix/tasks/protox/benchmark · high confidence

Removal of legacy config/config.exs file

The project has removed the main \config/config.exs\ file, which previously contained boilerplate comments and configuration for dependencies like \protox\ and \logger\. This change indicates a shift in how application configuration is managed, likely moving towards environment-specific config files or configuration defined directly in \mix.exs\, as suggested by the comments in the deleted file.

config · high confidence

Standardized development environment with linting, formatting, and static analysis

The project now enforces consistent code quality through a new configuration suite: \.credo.exs\ defines strict linting rules (with specific exclusions for generated code in \lib/protox/parse.ex\), \.formatter.exs\ sets a 120-character line length and integrates the Quokka plugin, and \.dialyzer\_ignore.exs\ suppresses known false positives in test support files. These configurations are documented in the new \AGENTS.md\ and \CONTRIBUTING.md\ guides, which also specify the required toolchain (Elixir ≥ 1.15, OTP ≥ 26) and development commands for linting, formatting, and static analysis.

(repo-wide) · high confidence

Test coverage

Added property-based test infrastructure using StreamData; Added sample .proto files for testing; Expanded test coverage for Protox decoding, encoding, and generation; Expanded test coverage for code generation, conformance, and protocol features.

Dependencies

Major dependency overhaul and Elixir 1.15 requirement

The project has been upgraded to require Elixir 1.15 and significantly updated its development and testing dependencies. Key changes include bumping Credo to 1.7.19, ExDoc to 0.40.4, Dialyxir to 1.4.7, and ExCoveralls to 0.18.5. The codebase also introduces Muex (0.11.2) for mutation testing, StreamData (1.4.0) for property-based testing, and Quokka (2.13.1) for code style enforcement, while removing older tools like Propcheck and Git Hooks. Additionally, the project now supports compiling against a specific version of the official Google protobuf library via the PROTOX\_PROTOBUF\_VERSION environment variable.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Baseline

  • First survey — no prior run to compare against. CAI 79.

Lenses

  • Code Health 98
  • Architecture 100
  • Maturity 62
  • Readiness 93
  • Security 93

Changes since last survey

  • 300 commits — 280 feature/other, 20 fixes

By area

  • (root) — 114 commits
  • lib/protox — 110 commits
  • .github/workflows — 26 commits
  • test/protox — 10 commits
  • lib/google — 5 commits
  • test/support — 5 commits
  • .config/mise.toml — 4 commits
  • lib/mix — 4 commits
  • benchmark/mix — 3 commits
  • benchmark/protos — 2 commits
  • documentation/reference.md — 2 commits
  • lib/protox.ex — 2 commits
  • .config/lefthook.yml — 1 commit
  • .github/dependabot.yml — 1 commit
  • .safe/config.json — 1 commit
  • .safe/ignore.json — 1 commit
  • benchmark/README.md — 1 commit
  • benchmark/benchmark_payloads.bin — 1 commit
  • benchmark/launch_benchmark.md — 1 commit
  • conformance/failure_list.txt — 1 commit

Notable commits

  • fix: chore: fix comment about ignored fields
  • fix: fix(encoder): improve error attribution for map fields during encoding (#313)
  • fix: fix: Add key validation to decoder
  • fix: fix: Use + for mix do
  • fix: fix: add support for nested extensions in
  • fix: fix: avoid missing modules warnings
  • fix: fix: avoid quadratic unknown field decode cost (#280)
  • fix: fix: benchee is needed at runtime in dev and test
  • fix: fix: enforce length-delimited wire type checks in decoder (#281)
  • fix: fix: format code
  • fix: fix: handle FileOptions containing messages
  • fix: fix: improve decoding of map entries with invalid wire types
  • fix: fix: keep order of unknown fields
  • fix: fix: reject wrong-typed message values and make error attribution total
  • fix: fix: remove quadratic decoding of repeated elements. (#287)
  • fix: fix: resolve MODULE in use Protox options (#278)
  • fix: fix: the first element of a oneof type is the name of the set field
  • fix: fix: tighten encoding error attribution after PR review
  • fix: refactor: revert to an accumulator to encode a message
  • fix: test: fix invalid double test
  • …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

ahamez/protox was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 18 September 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 702c78895e90099bed83d581b9f6ef46956c077e — the exact code this score is about.
  • Scored under rubric-2026.09.15 — the same rubric and the same method as every other entry in this index.
  • Measured by watchdog.canine.dev using codehealth-analyzer preprod-5d04157a340d.