basho/riak_cs

This system is a distributed object storage service compatible with the Amazon S3 API, built on the Erlang-based Riak platform. It provides core storage, garbage collection, and access logging capabilities, while also offering administrative and diagnostic tools for cluster management and data repair. The codebase includes a comprehensive test suite that validates S3 compatibility across multiple client libraries (Java, Python, PHP, Ruby, and Clojure) and verifies internal system behaviors such as garbage collection and access statistics.

13 changes.

This period focused on modernizing the project's operational and testing infrastructure. The team introduced Exometer for metrics collection and established a new Rel/ build system with Cuttlefish configuration management. Additionally, they significantly expanded test coverage through new automated suites for S3 API compatibility, client interactions, and upgrade scenarios.

9 changes.

This period focused on significantly expanding automated test coverage for the Riak CS codebase, particularly through the introduction of new client test suites for PHP, Python, and Ruby. The work involved adding comprehensive unit and integration tests for S3 API operations, including authentication, bucket management, and object storage. Additionally, internal diagnostic tools and dependency management utilities were added to support testing and project structure visualization.

4 changes.

This period focused on enhancing the development environment and testing capabilities. A new configuration setup for Riak CS was introduced to support local development, while a repair utility was added to handle invalid state manifests in the garbage collection bucket. Additionally, the project expanded its testing framework by importing an S3 client library and adding intercepts to simulate error conditions.

Added a new PHP test suite for the AWS SDK, including a Makefile to manage dependencies and run tests, a PHPUnit configuration, and a test services configuration file. The tests are designed to work with aws-sdk-php-2.3.x and require PHP 5.4.x with Composer for dependency management.

_client\tests/php · high confidence

A new Rebar plugin, rebar\_test\_plugin.erl, has been added to the .plugins directory. This plugin introduces support for running client tests and Riak tests via Rebar, including compilation and execution of EUnit tests. The plugin also includes a check for the presence of the EQC library to conditionally include EQC-related compilation flags.

.plugins · high confidence

A new automated test suite for the Python boto client has been added to the client\_tests/python/boto\_tests directory. The suite includes a test script (boto\_test.py) and a helper module (file\_generator.py) that supports both AWS Signature v2 and v4 authentication mechanisms. To ensure correct behavior with AWS Signature v4, the test environment uses a patched version of boto's auth.py (auth.py.patched), which applies a specific patch (bb5e02b1c61b3cf03ba9cafc7a3f6c56adcf92ad.patch) to fix known bugs in the original library's handling of header canonicalization and URL encoding.

_client\_tests/python/boto\tests · high confidence

A new Erlang script, misc/mapdeps.erl, has been added to map a project's dependencies by parsing rebar.config files. This utility generates a Graphviz-compatible directed graph of the dependency tree, allowing users to visualize the project's structure and dependency relationships.

misc · high confidence

New intercept modules have been added to the test suite to simulate specific error conditions, including timeouts and overloads. The \intercept\ module provides a framework for wrapping target modules (such as \riak\_cs\_block\_server\, \riak\_cs\_riak\_client\, and \riakc\_pb\_socket\) to inject failures like \timeout\ or \overload\ errors, enabling tests to verify how the system handles these scenarios.

_riak\test/intercepts · high confidence

The \priv/tools/internal\ directory now includes several new Erlang scripts for internal use: \block\_audit.erl\ and \ensure\_orphan\_blocks.erl\ to scan for and verify orphaned blocks; \offline\_delete.erl\ to directly delete stale blocks from bitcask databases; \select\_gc\_bucket.erl\ to identify large deleted manifests in the GC bucket; and \riak\_cs\_inspector.erl\ to inspect Riak CS data via the PB API. A README.md file has been added to document these internal tools and their use cases.

priv/tools/internal · high confidence

A new escript tool, \repair\_gc\_bucket.erl\, has been added to the \priv/tools\ directory. This utility scans the Riak CS garbage collection (GC) bucket to identify and repair manifests with invalid states. It connects to a Riak cluster via the PBC interface, retrieves manifests, and updates their state to \pending\_delete\ if they are stuck in an invalid condition, thereby allowing the garbage collection process to proceed correctly.

priv/tools · high confidence

Three new shell scripts are introduced in the riak\_test/bin directory to facilitate upgrade testing. rtdev-build-releases.sh automates the download and compilation of specific Riak CS versions (e.g., 1.5.4) using designated Erlang runtimes. rtdev-current.sh manages the 'current' release by extracting version metadata and updating the destination directory. rtdev-setup-releases.sh creates a mixed-version directory structure from existing devrels, enabling the testing of upgrade scenarios across different Riak CS versions.

_riak\test/bin · high confidence

The Ceph S3 test suite is now integrated into the automated client tests, allowing S3 compatibility tests to be run as part of the standard test suite. The setup includes a Makefile to manage dependencies (cloning the s3-tests repository and installing requirements) and a configuration script that generates the necessary S3 test configuration by fetching user credentials from the Ceph server. This enables running a broader range of S3 API tests against Ceph.

_client\_tests/python/ceph\tests · medium confidence

The \rel/files\ directory now includes a new \riak\_cs.schema\ file that defines the Cuttlefish configuration schema for Riak CS, replacing the previous \advanced.config\ and manual config generation. This change introduces a structured way to manage settings such as connection pools, garbage collection intervals, and admin listeners. Additionally, new shell scripts are added: \riak-cs-admin\ serves as the primary interface for administrative tasks (status, garbage collection, storage, stanchion, and cluster info), while \riak-cs-debug\ provides a comprehensive troubleshooting tool for gathering logs and system information. Several legacy scripts (\riak-cs-access\, \riak-cs-gc\, \riak-cs-storage\, \riak-cs-stanchion\, \riak-cs-supercluster\) are updated to display deprecation warnings and redirect to the new \riak-cs-admin\ or \riak-cs-supercluster\ commands.

rel/files · high confidence

Adds new modules (riak\_cs\_access, riak\_cs\_access\_log\_handler, riak\_cs\_access\_archiver, and riak\_cs\_access\_console) to collect and archive I/O usage statistics. The system now tracks access logs, flushes them to Riak periodically, and provides a command-line interface (riak\_cs\_access\_console) to manually trigger flushing and monitor the archiving process.

src · high confidence

Added a new test harness for Riak CS (Riak CS and Stanchion) in the \riak\_test\ suite. This includes helper modules for managing CS nodes, configuring test environments, and executing operations like user creation, object listing, and replication. The changes introduce new files such as \cs\_suites.erl\, \rtcs.erl\, \rtcs\_admin.erl\, \rtcs\_config.erl\, \rtcs\_dev.erl\, and \rtcs\_exec.erl\, which collectively provide the infrastructure for running CS-specific tests, including setup/teardown, configuration management, and execution of CS-related test cases.

_riak\test/src · high confidence

Added new Erlang header files that define records and types for the S3 API (including bucket and object actions, policies, and conditions), object listing requests and responses, garbage collection states, usage tracking fields, and OpenStack/Keystone authentication requests. These headers establish the data structures and constants used by the storage system to handle S3-compatible operations, background cleanup, and usage statistics.

include · high confidence

The rel/ directory now contains the complete build and release configuration for Riak CS 2.1.0. This includes a new gen\_dev script for generating development environment configurations, a rebar.config that integrates Cuttlefish, and a reltool.config that defines the application dependencies (including exometer\_core, cluster\_info, and riak\_cs) and file overlays. Additionally, a new vars.config file provides default settings for network ports, authentication, and Cuttlefish integration, enabling the creation of standalone release packages and development environments.

rel · high confidence

A new development configuration file (dev\_vars.config.src) is introduced, defining runtime variables for a Riak CS setup. It configures platform paths, connection details for CS, Admin, Riak, and Stanchion (including SSL settings), and enables Cuttlefish for configuration management. This change supports local development and testing of the CS nodes.

rel/vars · medium confidence

The S3 client implementation in client\_tests/clojure/clj-s3/src/aws/sdk/s3.clj has been replaced with an import from the clj-aws-s3 library (tag 0.3.10). This introduces a comprehensive set of functions for interacting with Amazon S3, including bucket management (create, delete, list), object metadata handling, and support for AWS credentials and proxy configurations.

_client\tests/clojure/clj-s3/src/aws · medium confidence

Riak CS has migrated its metrics collection from Folsom to Exometer, enabling new system health and storage usage metrics. Additionally, several dependencies have been updated, including cuttlefish, node\_package, riak-erlang-client, lager, and others, to support the new metrics system and ensure compatibility.

(repo-wide) · high confidence

The apps/riak\_cs directory now uses symbolic links to point to shared build artifacts and source code. Specifically, the ebin, include, priv, and src directories within apps/riak\_cs are now symlinks to their counterparts in the parent directory. This change aligns the project structure with a centralized build and source layout, ensuring that Riak CS shares the same directory conventions as the rest of the codebase.

_apps/riak\cs · high confidence

Added Clojure-based S3 client tests for the Java SDK; Added Clojure-based compatibility tests for the Java S3 SDK; Added PHP unit tests for S3 client operations; Added QuickCheck-based test for erlcloud S3 API interactions; Added Ruby client tests for S3 operations; Expanded automated test coverage for Riak CS components; Expanded test coverage for Riak CS features.

The project now includes dependency manifests for client-side integration tests: a composer.json and composer.lock for the PHP tests (pinning aws-sdk-php 2.3.4 and its dependencies), a requirements.txt for the Python boto tests (pinning boto 2.35.1), and a Gemfile for the Ruby tests (pinning aws-sdk 1.8.5, uuid 2.3.6, rake 10.0.2, httparty 0.10.2, and rspec 2.11.0). These changes ensure that each language-specific test suite can be installed and run with consistent, locked versions of the required libraries.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 45 → 45 (-0.3)
• Rubric changed (rubric-2026.08.18 → rubric-2026.08.19) — scores are not directly comparable.

• Code Health 92 → 92 (+0.0)
• Architecture 100 → 98 (-2.1)
• Maturity 42 → 42 (-0.8)
• Readiness 18 → 18 (+0.0)
• Security 90 → 90 (+0.0)
• Event Sourcing 100 → 100 (+0.0)

• LLM evaluation failed

• Change coupling: riak_cs_gc_batch.erl ↔ riak_cs_gc_console.erl (src/riak_cs_gc_batch.erl)
• Change coupling: riak_cs_wm_bucket.erl ↔ riak_cs_wm_object.erl (src/riak_cs_wm_bucket.erl)
• Change coupling: riak_cs_wm_user.erl ↔ riak_cs_wm_users.erl (src/riak_cs_wm_user.erl)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

basho/riak_cs was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.