microsoft/FASTER

This system is FASTER, a high-performance, distributed key-value store and append-only log library available in C\# and C++. It provides a session-based API for managing data across hot in-memory and cold on-disk storage tiers, supporting features like automatic compaction, checkpointing, and recovery. The system enables remote access via TCP and WebSocket protocols, allowing clients to perform standard CRUD operations, read-modify-write transactions, and pub-sub messaging.

28 changes.

This period focused on introducing the F2 key-value store with hot/cold tiering and automatic compaction, alongside a comprehensive refactoring of the C\# API to a session-based model. The work also involved modernizing the C++ build system with CMake and C++17, adding Azure Blob and io\_uring support, and removing legacy code such as runtime code generation and mixed-mode layers.

19 changes.

This period focused on refactoring FASTER's core persistence and synchronization mechanisms into modular, pluggable architectures, introducing a state machine for checkpointing and a flexible client session API with async support. Significant enhancements included native support for variable-length types, the addition of Azure Page Blob storage for tiered storage, and comprehensive sample applications demonstrating these new capabilities across various scenarios.

24 changes.

This period focused on extending FASTER's capabilities by introducing a comprehensive remote client-server architecture, including TCP and WebSocket protocols, JavaScript support, and pub-sub messaging. Concurrently, the core engine underwent significant internal restructuring, modularizing the index implementation, refining async operation completion logic, and adding features like log compaction and configurable commit policies.

A new C\# sample project (EpvsSample) has been added to demonstrate the Epoch Protected Version Scheme (EPVS) concurrency primitive. The sample includes a benchmarking tool (EpvsBench) that compares EPVS against latch-based synchronization across multiple threads and NUMA configurations, alongside example implementations (ListExample) showing how to use EPVS for thread-safe resizable lists.

cs/samples/EpvsSample · high confidence

A new sample application has been added to the cs/playground/ClassRecoveryDurability directory to demonstrate durable key-value storage using the FASTER library. The sample includes a Storedb class for initializing and recovering the database, custom types for keys and values with binary serialization, and a Program entry point that performs continuous insert and delete operations with periodic checkpoints to ensure data durability.

cs/playground/ClassRecoveryDurability · high confidence

A new benchmark executable, \benchmark\_f2\, is introduced to evaluate the F2 key-value store, which utilizes a hybrid log architecture with separate hot and cold indexes and logs. This addition includes the \benchmark.h\ header defining the workload types (such as 50:50 read/write and 100% RMW) and context classes, alongside \benchmark\_f2.cc\ which configures the F2 store with specific memory and disk settings. The CMake build system is updated to compile this new benchmark target, and the README is expanded with usage instructions for running YCSB-style workloads against the new store.

cc/benchmark-dir · high confidence

Introduces the SumStore playground example, which demonstrates the FASTER session-based interface using C\# structs for keys and values. The implementation defines an AdId key and NumClicks value, along with a Functions class that handles read operations and read-modify-write (RMW) logic for aggregating click counts, including checkpoint completion callbacks.

SumStore · high confidence

The VarLenClient sample now demonstrates how to connect to a remote FASTER server using variable-length keys and values. It includes examples for custom struct types (CustomType) and .NET ReadOnlyMemory buffers, covering both synchronous and asynchronous operations such as upsert, read, delete, and pub-sub subscriptions. The sample also provides a memory benchmark to measure performance with batched reads.

cs/remote/samples/VarLenClient · high confidence

A new C\# sample project has been added to demonstrate FASTER usage on .NET Framework 4.6. The sample includes an App.config targeting v4.6, a Program.cs illustrating in-memory and disk-based key-value operations (including upsert, delete, and read-modify-write with summation), and standard assembly metadata.

cs/samples/HelloWorld · high confidence

A new C\# playground application (AsyncStress) has been added to benchmark and stress-test the FASTER key-value store under various asynchronous workloads. The tool provides a command-line interface to configure storage types (blittable value types, reference types with object logs, and SpanByte serialization via MessagePack), threading modes (single-threaded, Parallel.For, or chunked tasks), and operational modes (Upsert, Read-Modify-Write, or mixed). It includes wrapper implementations (FasterWrapper and SerializedFasterWrapper) that manage session pooling and async completion, allowing users to profile performance characteristics like pending operation counts and tail addresses across different configuration scenarios.

cs/playground/AsyncStress · high confidence

The core library now includes the F2 key-value store, which manages data across a hot in-memory hybrid log and a cold on-disk hybrid log. This change introduces the F2Kv orchestrator, a ColdIndex for on-disk hash storage, and dedicated internal contexts to handle multi-stage read and read-modify-write operations across both tiers. To maintain performance and storage efficiency, the store features automatic background compaction for both the hot and cold logs, configurable via TOML settings, and includes a new read cache mechanism to accelerate access to frequently used records.

cc/src/core · high confidence

This change adds the \Microsoft.FASTER.Client\ and \Microsoft.FASTER.Server\ NuGet packages, enabling FASTER to be used as a remote key-value store accessible over the network. The server package supports .NET Standard 2.0/2.1, .NET 6, and .NET 7, while the client package targets .NET Standard 2.0/2.1, .NET 6, and .NET 7. The server package depends on \Microsoft.FASTER.Core\ and \Microsoft.Extensions.Logging\ (version 3.1.26 for older frameworks, 7.0.0 for .NET 6/7), and the client package depends on \System.Runtime.CompilerServices.Unsafe\ and \System.Memory\. A new solution file (\FASTER.remote.sln\) and sample projects (VarLen/FixedLen Client/Server, WebClient) are included to demonstrate usage.

cs/remote · high confidence

The FasterLog component now provides a dedicated append-only log implementation featuring configurable commit behaviors via the LogCommitPolicy abstraction (Default, MaxParallel, and RateLimit modes), allowing users to control commit latency and throughput. It supports automatic commit and safe-tail refresh settings, async iteration with ILogEntryConsumer and IBulkLogEntryConsumer interfaces, and introduces CommitFailureException for detailed error handling during commit operations.

cs/src/core/FasterLog · high confidence

Adds the \FASTER.client\ library, enabling C\# applications to connect to and interact with a remote FASTER KV store over TCP. This new component provides the \FasterKVClient\ entry point and \ClientSession\ for executing operations (Upsert, Read, RMW, Delete) both synchronously and asynchronously, along with built-in support for fixed-length and variable-length (Memory) data serialization and pub-sub capabilities.

cs/remote/src/FASTER.client · high confidence

This change adds the foundational networking and serialization components for remote FASTER operations. It introduces a \BatchHeader\ struct for packing sequence numbers and wire protocols, along with \BufferSizeUtils\ to calculate optimal client and server buffer sizes based on \MaxSizeSettings\. New interfaces (\IClientSerializer\, \IServerSerializer\, \IKeySerializer\, \IMessageConsumer\) define the contract for serializing keys, values, and inputs across the network. The \TcpNetworkSender\ implements the actual transport layer, featuring a \SimpleObjectPool\ for reusing \SeaaBuffer\ instances and a configurable \ThrottleMax\ semaphore to limit concurrent outstanding sends. Additionally, \MessageType\ and \WireFormat\ enums define the supported operations (Read, Upsert, RMW, Delete, Pub/Sub) and serialization formats (DefaultVarLenKV, DefaultFixedLenKV, WebSocket, ASCII).

cs/remote/src/FASTER.common · high confidence

This change introduces the foundational server-side components for the FASTER remote API, including the \FasterServerBase\ abstract class for session and provider management, the \FasterServerTcp\ implementation for handling TCP connections, and the \GenericServer\, \FixedLenServer\, and \VarLenServer\ classes for configuring and running the key-value store. It also adds the \ServerOptions\ configuration class and the \IFasterServer\ interface, establishing the new structure for server initialization, network handling, and resource disposal.

cs/remote/src/FASTER.server/Servers · high confidence

The core memory allocation subsystem for the hybrid log has been expanded with a new set of base classes and specialized implementations. This includes the abstract AllocatorBase defining the core address management and epoch handling, alongside concrete BlittableAllocator and GenericAllocator classes to handle fixed-size and variable-size records respectively. To support these, new frame structures (BlittableFrame, GenericFrame) manage the in-memory circular buffers, and dedicated scan iterators (BlittableScanIterator, GenericScanIterator, MemoryPageScanIterator) enable efficient traversal of the log. The change also introduces supporting infrastructure such as AsyncIOContext for asynchronous operations, AtomicOwner for lock-free queue management, and ErrorList for tracking commit errors.

cs/src/core/Allocator · high confidence

FASTER now supports log compaction, allowing users to reclaim space by moving active records to the tail of the log and shifting the begin address. The new \Compact\ method accepts a \CompactionType\ to choose between \Scan\ (checking record liveness by scanning the log) and \Lookup\ (checking liveness using the hash chain). Users can provide custom \ICompactionFunctions\ to define additional deletion logic, such as handling reference-counted records.

cs/src/core/Compaction · high confidence

FASTER now supports using System.Memory\<T\> and SpanByte as keys and values directly, eliminating the need for manual serialization. This change introduces a new \VarLen\ component layer featuring \IVariableLengthStruct\ interfaces and concrete implementations for \SpanByte\, \Memory\<T\>\, and \ReadOnlyMemory\<T\>\. It also provides \MemoryFunctions\ and \SpanByteFunctions\ to handle read, write, and read-modify-write operations, along with \MemoryComparer\ and \SpanByteComparer\ for equality checks, allowing users to store variable-length data in-place on the log.

cs/src/core/VarLen · high confidence

Added a new \AzureStorageDevice\ implementation that uses Azure Page Blobs to provide scalable, shared cloud storage for FASTER's tiered storage feature. This device supports configurable lease management for concurrent access, automatic retry logic for transient errors, and integrates with the existing \INamedDeviceFactory\ interface for seamless integration into FASTER's log and key-value store architectures.

cs/src/devices · high confidence

Added a new C\# sample application in cs/remote/samples/FixedLenServer (and its client counterpart) that demonstrates how to use FASTER over a remote connection with fixed-length (blittable) 8-byte keys and values. The server implementation uses a LockTable concurrency control mode and ILogger-based logging, while the client showcases both synchronous and asynchronous APIs for upsert, read, delete, and read-modify-write (RMW) operations, including subscription capabilities.

cs/remote/samples/FixedLenServer · high confidence

Added new sample projects demonstrating FASTER usage: CacheStore illustrates a key-value store with optional read caching, custom serializers, and checkpointing, while FasterLogSample demonstrates synchronous and asynchronous log writing, scanning, and truncation. These samples provide concrete examples of integrating FASTER into .NET applications for high-performance data storage and streaming.

cs/samples/CacheStore, cs/samples/FasterLogSample · high confidence

Adds a new command-line stress testing tool for FASTER that exercises the store with configurable workloads. The application supports Long, String, and SpanByte key and value types, and allows tuning of thread count, operation mix (Read, Upsert, Delete, RMW), and locking behavior (including Lockable Unsafe Context). It also includes support for periodic checkpoints and log compaction to validate durability and storage management under load.

cs/stress · high confidence

A new sample application has been added to the cs/samples/FasterLogPubSub location that illustrates a publish-subscribe pattern using FasterLog. It demonstrates two modes: a single-instance mode where a producer, consumer, and committer operate on the same log, and a separate-instance mode where a secondary read-only FasterLog continuously recovers and catches up with the primary log's commits via RecoverReadOnlyAsync. This sample serves as a practical guide for implementing log replication and cross-process consumption scenarios.

cs/samples/FasterLogPubSub · high confidence

This change introduces a new JavaScript client library (FASTER.client/JavascriptClient) that enables browser-based applications to interact with a remote FASTER server over WebSockets. The library provides core networking and serialization components, including ClientSession for managing operations like Read, Upsert, Delete, and RMW, as well as new Subscribe and Publish capabilities for key-value and pub-sub messaging. A sample web application (WebClient) is included to demonstrate usage, featuring a UI for performing store operations and subscribing to key updates, with specific attention to security by rendering server responses as text to prevent DOM-based XSS.

cs/remote/samples/WebClient, cs/remote/src/FASTER.client/JavascriptClient · high confidence

Added two new PowerShell scripts, \run\_benchmark.ps1\ and \compare\_runs.ps1\, to the \cs/benchmark/scripts\ directory. \run\_benchmark.ps1\ automates the execution of FASTER benchmark permutations across multiple builds (supporting both pre-built binaries and cloning/building from git branches), generating structured result files. \compare\_runs.ps1\ analyzes these results to compare performance metrics like throughput and standard deviation between two directories of runs, helping users identify performance improvements or regressions.

cs/benchmark/scripts · high confidence

A new sample application, ReadAddress, has been added to demonstrate how to read the full history of versions for a specific key in FASTER. The sample populates a store with upserts and deletes, optionally using Read Cache and checkpoints, and then iterates through the version chain for a key to display each record's value and previous address, including tombstoned (deleted) entries.

cs/samples/ReadAddress · high confidence

A new sample application (StoreAsyncApi) has been added to demonstrate the async read-modify-write (RMW) API that returns a status. The sample showcases how to perform batched and single async upserts, handle pending statuses via CompleteAsync, and manage checkpoints asynchronously. It also includes updated type definitions (CacheKey, CacheValue) and functions (CacheFunctions) adapted for the new async interface, replacing the previous synchronous IFASTERKey/IFasterValue patterns with IFasterEqualityComparer and FunctionsBase.

cs/samples/StoreAsyncApi · high confidence

A new benchmark sample has been added to demonstrate disk-based read and write operations using FASTER. The sample includes a C\# program that initializes a FasterKV instance with a persistent log device, performs parallel asynchronous upserts and batched reads, and validates data integrity. It also provides the necessary type definitions (Key, Value, Input, Output) and callback functions (MyFuncs) to illustrate how to implement custom logic for single-reader, concurrent-reader, and read-modify-write operations within the benchmark context.

cs/samples/StoreDiskReadBenchmark · high confidence

A new sample application has been added to demonstrate log compaction via sessions. The sample initializes a FASTER KV store, performs bulk upserts, and then runs a continuous loop of mixed deletes and upserts. It explicitly calls \s.Compact\ with \CompactionType.Scan\ to reclaim space and truncates the log, allowing users to observe memory usage and log address progression. The sample includes updated type definitions (\CacheKey\, \CacheValue\, \CacheFunctions\) that implement the necessary interfaces for serialization and equality comparison within this context.

cs/samples/StoreLogCompaction · high confidence

The benchmark tool in \cs/benchmark\ has been reorganized and expanded to support three distinct workload modes: the standard FASTER KV store, a SpanByte-optimized FASTER variant, and a baseline \ConcurrentDictionary\. Users can now select the target via the \-b\ flag (0, 1, or 2) and configure operation mixes (reads, upserts, RMWs, deletes) using the new \-rumd\ argument. The suite also introduces command-line options for NUMA affinity (\-n\), concurrency control modes (\-z\), periodic checkpointing (\-chkptms\), and data recovery from checkpoints (\-k\), allowing for more granular performance testing across different hardware and configuration scenarios.

cs/benchmark · high confidence

The CheckpointManagement module now introduces a structured, pluggable architecture for handling index and log checkpoints. A new ICheckpointNamingScheme interface and its DefaultCheckpointNamingScheme implementation standardize how checkpoint files (such as snapshots, delta logs, and commit metadata) are named and organized into specific directories (e.g., 'index-checkpoints', 'cpr-checkpoints'). This is paired with an INamedDeviceFactory interface and concrete implementations like LocalStorageNamedDeviceFactory, which abstracts the underlying storage operations (get, delete, list) for these checkpoint files. The DeviceLogCommitCheckpointManager utilizes these abstractions to manage log commits and checkpoints, supporting features like automatic purging of outdated commits and throttled metadata writes.

cs/src/core/Index/CheckpointManagement · high confidence

Added a new C\# sample application in the CacheStoreConcurrent playground that demonstrates using FASTER as a multi-threaded cache and key-value store. The sample includes Program.cs and Types.cs, which define custom CacheKey and CacheValue types with binary serializers, configure FASTER KV instances with optional read caching and checkpointing, and run a continuous random read workload across multiple threads to measure throughput.

cs/playground/CacheStoreConcurrent · high confidence

The FASTER server now includes a new remote communication layer that allows clients to interact with the store over the network. This change introduces a provider-based session architecture (ISessionProvider) that supports two wire protocols: a custom binary protocol (BinaryServerSession) and WebSocket (WebsocketServerSession). The implementation includes server-side serializers for both fixed-length blittable types (FixedLenSerializer) and variable-length SpanByte types (SpanByteServerSerializer), along with corresponding client serializers and callback functions (ServerKVFunctions, SpanByteFunctionsForServer). Users can now expose FASTER instances remotely, supporting standard KV operations (Upsert, Read, RMW, Delete) and pub-sub capabilities via the new broker integrations.

cs/remote/src/FASTER.server · high confidence

Added the StoreVarLenTypes sample application, which demonstrates how to use FASTER with variable-length keys and values using SpanByte and Memory\<T\> types. The sample includes specific examples for performing Read-Modify-Write (RMW) operations on ASCII numbers using SpanByte, as well as storing and retrieving Memory\<byte\> and Memory\<int\> data structures.

cs/samples/StoreVarLenTypes · high confidence

Added new sample projects to demonstrate FASTER usage with Azure Blob Storage and local checkpoint recovery. The AzureBackedStore sample shows how to configure AzureStorageDevice for both the main log and object log, manage checkpoints via DeviceLogCommitCheckpointManager, and recover state from cloud storage. The StoreCheckpointRecover sample demonstrates local disk-based checkpointing and recovery using FasterKVSettings with TryRecoverLatest, including custom type serialization. The StoreCustomTypes sample illustrates handling non-blittable custom classes with BinaryObjectSerializer and manual device management.

cs/samples/StoreCheckpointRecover · high confidence

FASTER introduces a new session-based interface model, exposing \IFasterKV\ and \IFasterSession\ to manage client sessions, recovery, and checkpointing. To support fine-grained control over record updates, new callback info structs (\UpsertInfo\, \RMWInfo\, \DeleteInfo\) and action enums (\UpsertAction\, \RMWAction\, \DeleteAction\) are added, allowing users to cancel operations or expire records during RMW callbacks. The update also includes \FunctionsBase\ and \SimpleFunctions\ to simplify user implementations, \IFasterEqualityComparer\ with optimized comparers for common types, and a new \DataContractObjectSerializer\ for class serialization.

cs/src/core/Index/Interfaces · high confidence

FASTER introduces a new \ClientSession\ API that provides a thread-independent interface for database operations, allowing sessions to be suspended and resumed across different threads. This new session model includes full \async/await\ support for \Read\ and \Upsert\ operations, enabling non-blocking I/O handling. Additionally, it introduces a \LockableContext\ mechanism, allowing applications to manually manage shared and exclusive locks on keys to prevent deadlocks and ensure consistency during concurrent updates.

cs/src/core/ClientSession · high confidence

A new sample server for variable-length keys and values has been added to the remote samples. It introduces a command-line interface for configuring server parameters such as port, memory size, page size, segment size, index size, and storage tiering. The server supports pub/sub features (configurable via flags), checkpoint recovery, and uses Microsoft.Extensions.Logging for console output.

cs/remote/samples/VarLenServer · high confidence

The checkpoint and recovery subsystem has been refactored to use a pluggable \ICheckpointManager\ interface, allowing users to control the creation, storage, and retrieval of checkpoint data (including index, hybrid log, snapshot, and delta logs) via custom implementations. This change introduces new internal components in the \Recovery\ namespace—such as \Checkpoint.cs\, \DeltaLog.cs\, \DirectoryConfiguration.cs\, and \IndexRecovery.cs\—to manage checkpoint metadata, file descriptors, and asynchronous recovery operations, enabling more flexible and customizable persistence strategies.

cs/src/core/Index/Recovery · high confidence

The ResizableCacheStore sample now includes a new CacheSizeTracker and LogSizeTracker that monitor and dynamically adjust the memory footprint of the FASTER store. By subscribing to eviction notifications, the sample automatically resizes the main log and optional read cache to stay within a specified target memory limit, providing users with a concrete example of how to implement and manage dynamic memory constraints in a production-like cache scenario.

cs/samples/ResizableCacheStore · high confidence

Added a new sample application in cs/samples/SecondaryReaderStore that illustrates how to configure a read-only secondary FasterKV store to continuously recover and catch up with a primary store. The sample runs two concurrent threads: a primary writer that periodically takes snapshot checkpoints and a secondary reader that repeatedly attempts recovery to read the latest data, demonstrating the mechanism for keeping a secondary store synchronized with primary updates.

cs/samples/SecondaryReaderStore · high confidence

The server now implements the core logic for handling pub-sub subscriptions and broadcasting messages to subscribers. This includes new serializers for fixed-length and variable-length (SpanByte) keys, glob-style pattern matching utilities for prefix subscriptions, and two broker classes (SubscribeBroker and SubscribeKVBroker) that manage subscription lifecycles, log scanning, and message distribution to connected sessions.

cs/remote/src/FASTER.server/PubSub · high confidence

The FASTER.sln solution file and all associated playground demonstration projects (including ManagedSample1-4, ClassCache, SumStore, and NestedTypesTest) have been removed from the cs/src directory. This eliminates the example codebases that previously illustrated various usage patterns of the FASTER core library, such as unsafe pointer-based operations, managed IUserFunctions implementations, and complex nested type handling.

cs/src, cs/src/playground · high confidence

Deleted the \cs/src/test\ directory, removing the legacy FASTER test suite. This includes \BasicFASTERTests\ (native in-memory write/read and RMW operations), \ObjectFASTERTests\ (managed object store operations), \ComponentRecoveryTests\ (allocator and fuzzy index recovery), \FullRecoveryTests\ (full checkpoint and recovery workflows), and the associated test type definitions (\TestTypes.cs\, \RecoveryTestTypes.cs\).

cs/src/test · high confidence

The legacy Mixed-mode managed layer implementation has been removed from the core library. This change deletes the \FASTERFactory\ helper, the \IManagedFAST\ and \IFASTER\_Mixed\ interfaces, and all associated wrapper types (such as \MixedKeyWrapper\, \MixedValueWrapper\, and \MixedFunctionsWrapper\) that previously facilitated mixed-mode operations. Users relying on this specific managed abstraction for key-value operations will no longer have access to these components.

cs/src/core/ManagedLayer · high confidence

The runtime code generation subsystem in \cs/src/core/Codegen\ has been removed. This change deletes the Roslyn-based compilers (\CompilerBase\, \FasterHashTableCompiler\, \MixedBlitManagedFasterHashTableCompiler\) and their supporting utilities (\RoslynHelpers\, \TypeReplacer\, \Utilities\) that previously generated C\# source code and compiled it into assemblies at runtime. Consequently, the \HashTableManager\ class, which served as the factory for creating FASTER hash table instances via this dynamic compilation, is also removed. Users can no longer rely on runtime code generation to create hash table instances; this functionality must be provided by other parts of the system or replaced by pre-compiled alternatives.

cs/src/core/Codegen · high confidence

The \adv-file-ops\ and \readtsc\ native C++ projects have been removed from the codebase. These libraries provided Windows-only functionality, including advanced file operations (such as setting file size and volume privileges via Win32 APIs) and high-resolution timestamp reading (via the RDTSC instruction). With these native dependencies removed, the core project no longer requires C++ toolchains or Windows-specific headers to build, supporting the move toward AnyCPU compatibility.

cs/src/native · high confidence

The internal implementation of the FASTER key-value index has been reorganized from monolithic files into distinct, focused modules to improve maintainability and clarity. This change introduces new source files for specific responsibilities: BlockAllocate.cs handles record allocation and retry logic; ConditionalCopyToTail.cs manages conditional record insertion; ContainsKeyInMemory.cs implements in-memory key existence checks; ContinuePending.cs processes pending asynchronous read and RMW operations; EpochOperations.cs manages epoch synchronization and waiting; FindRecord.cs contains the core record traversal and tracing logic; HandleOperationStatus.cs centralizes operation status handling and retry flows; HashEntryInfo.cs defines the hash table entry metadata structure; and Helpers.cs provides utility functions for record writing, version checking, and chain splicing. This modularization does not change the external API but restructures the internal codebase for better separation of concerns.

cs/src/core/Index/FASTER/Implementation · high confidence

The synchronization logic for checkpoints and index resizing has been restructured into a new, modular state machine framework located in \cs/src/core/Index/Synchronization\. This change introduces a \FasterStateMachine\ that orchestrates operations through a cycle of phases (REST, PREPARE, IN\_PROGRESS, etc.) and allows for pluggable \ISynchronizationTask\ implementations. Specifically, checkpointing is now handled by dedicated state machines like \FullCheckpointStateMachine\ and \IndexSnapshotStateMachine\, which coordinate index and hybrid log persistence in parallel, while index resizing is managed by \IndexResizeStateMachine\. This refactoring improves stability by ensuring strict synchronization between threads during state transitions, preventing issues such as race conditions during concurrent checkpointing and session starts, and providing a cleaner separation of concerns for future extensibility.

cs/src/core/Index/Synchronization · high confidence

The C++ build system has been updated to require CMake 3.14 and C++17 (or C++20 on MSVC), replacing the legacy ExternalProject approach with FetchContent for dependencies like Google Test (v1.14.0) and the newly added TOML11 (v4.3.0) configuration library. This change introduces two new optional features: an io\_uring-based I/O handler (enabled via USE\_URING) and Azure Blob Store support for the hybrid log (enabled via USE\_BLOBS), which requires linking against specific Azure and Boost libraries. Additionally, a helper script for installing Azure dependencies on Linux has been added, and the README has been simplified to point to external documentation.

cc · high confidence

The playground's sum-store examples have been updated to support a split checkpoint architecture where the index and hybrid log are managed separately. Checkpoint and recovery operations no longer rely on integer version numbers; instead, they use GUID tokens to identify specific checkpoint states. The \Checkpoint\ method now returns a token, and \Recover\ and \Continue\ methods accept index and hybrid-log tokens as arguments. Additionally, the persistence callback for checkpoints now receives a status result, allowing users to detect and report checkpoint failures. A new \recovery-info\ tool has been added to read and display metadata from index and log files, and the key hashing implementation has been updated to use \FasterHashHelper\.

cc/playground · high confidence

This change refactors the FASTER core API to support a session-based model and improve configuration and error handling. It introduces new configuration classes (CheckpointSettings, FasterKVSettings, LogSettings, SerializerSettings, ReadCacheSettings) to centralize hybrid log and checkpoint options, replacing scattered parameters. The CompletedOutputIterator and CompletedOutput structures are added to manage outputs from pending asynchronous operations, allowing clients to iterate over results and properly dispose of key/input containers. The AddressInfo struct is updated to use IntPtr for word storage, introduces a multiplier for larger object sizes, and throws FasterException on overflow. Additionally, the old Layout, Functions, Key, and Value structs (which used unsafe code and preprocessor directives for fixed/variable sizes) are removed, indicating a shift towards a more generic, type-safe, and session-oriented interface.

cs/src/core/Index/Common · high confidence

The FASTER store API has been refactored to require client sessions for iteration, removing the obsolete store-level Iterate methods that previously threw exceptions. A new Iterate API is now available on sessions, supporting both pull-based scanning and push-based iteration with custom scan functions. Additionally, the LogAccessor class now exposes the log as an observable, allowing users to subscribe to records as they become read-only or are evicted from memory, and extension methods provide an IObservable wrapper for scan iterators.

cs/src/core/Index/FASTER · high confidence

The C\# project structure has been reorganized to improve clarity and usability. The solution file now explicitly includes new playground projects such as MemoryDb, AsyncStress, and FasterLogStress, alongside updated sample projects like EpvsSample and ResizableCacheStore. This change provides developers with fresh, concrete examples of how to use FASTER components, including a new MemoryDb playground that demonstrates a multi-index database pattern and a FasterLogStress test for validating log performance.

cs · high confidence

The FASTER benchmark tool has been migrated from a local execution model to a remote client architecture. The benchmark application now connects to a remote FASTER server instance via configurable IP address and port (defaulting to 127.0.0.1:3278) instead of hosting the store locally. This change introduces command-line options for specifying the remote server address, port, and whether to skip the initial setup phase, enabling distributed performance testing scenarios.

cs/remote/benchmark · high confidence

The FASTER core NuGet package has been renamed from 'FASTER' to 'Microsoft.FASTER.Core' and is now authored by Microsoft. The library has expanded its target frameworks to include .NET 6.0 and .NET 7.0 (while dropping the previous .NET Framework 4.6 and .NET Core 3.1 targets), and the build output is now compiled for AnyCPU instead of x64-only. Additionally, the package now includes XML documentation files for all supported frameworks and updates its dependencies to align with the respective .NET versions, such as using Microsoft.Extensions.Logging 7.0.0 for .NET 7.

cs/src/core · high confidence

The locking implementation in the FASTER index has been restructured to manage concurrency via hash-bucket latches rather than previous mechanisms. This change introduces a new \ILockTable\ interface and specific implementations like \OverflowBucketLockTable\ to handle both manual and transient locking, supporting the new \ConcurrencyControlMode\ (replacing the older \LockingMode\). For users, this represents a behavioral change in how the index manages concurrent access, potentially affecting performance characteristics and lock acquisition strategies for shared and exclusive operations.

cs/src/core/Index/FASTER/Implementation/Locking · high confidence

On Linux, the file environment now supports an asynchronous I/O handler based on io\_uring (liburing), available when the FASTER\_URING build flag is enabled, alongside the existing libaio handler. The File class destructor now explicitly logs a warning if closing a file fails, and File::Delete always returns Status::Ok on success. Path handling is improved with new NormalizePath and NormalizeAndCreatePath utilities that ensure directory paths end with a separator and create missing directories. Additionally, internal types like Status, IAsyncContext, and AsyncIOCallback are now explicitly qualified with the core namespace to avoid ambiguity.

cc/src/environment · high confidence

The device storage layer has been refactored to use the \core::Status\, \core::AsyncIOCallback\, and \core::IAsyncContext\ types instead of the previous global \Status\ and callback types, ensuring consistent error handling and asynchronous I/O interfaces across the system. Additionally, a new \BlobFile\ class has been introduced to support storing data as Azure page blobs, enabling remote tiering to Azure Blob Storage, while local file operations in \FileSystemFile\ and \NullFile\ have been updated to align with the new core types and checkpoint path signatures.

cc/src/device · high confidence

The async execution model for Read, Upsert, Delete, and Read-Modify-Write (RMW) operations has been restructured to use a new \AsyncOperationInternal\ infrastructure. This change introduces dedicated \\*AsyncOperation\ and \\*AsyncResult\ structs for each operation type, separating the fast synchronous path from the slow asynchronous I/O path. Users will see more stable async behavior, particularly in low-memory conditions, as the new implementation correctly handles pending I/O completion and ensures that \CompletePending\ operations are properly awaited and completed without null reference exceptions.

cs/src/core/Async · high confidence

The core utilities have been refactored to improve async support and memory management. New asynchronous primitives, AsyncCountDown and AsyncQueue, have been added to support non-blocking wait and dequeue operations. The memory allocation system has been updated with a new SectorAlignedBufferPool and SectorAlignedMemory implementation that supports .NET 5+ GC allocation optimizations and configurable unpining on return. Additionally, the locking model has been expanded with a new ConcurrencyControlMode enum (supporting LockTable, RecordIsolation, and None) and associated locking utilities, while legacy async result types implementing IAsyncResult have been removed in favor of modern Task-based patterns.

cs/src/core/Utilities · high confidence

The epoch protection subsystem has been refactored to support a larger number of concurrent sessions and improve fine-grained scalability. The \LightEpoch\ class now uses a dynamic table size based on processor count and moves thread-static metadata into a separate \Metadata\ class to optimize cache usage. A new \FastThreadLocal\<T\>\ implementation provides a more robust mechanism for instance-thread-local variables, supporting up to 128 simultaneous instances per thread. Additionally, the \VersionSchemeState\ struct was introduced to manage version state machine phases and versions, enabling better handling of state transitions during operations like checkpoints.

cs/src/core/Epochs · high confidence

The server-side session provider architecture has been restructured to improve extensibility and type safety. A new abstract base class, FasterKVProviderBase, now handles common store initialization, recovery, and session creation logic for both WebSocket and Binary protocols. Concrete implementations like FasterKVProvider and SpanByteFasterKVProvider inherit from this base, allowing for more flexible configuration of functions and serializers. This change simplifies the creation of custom session providers and ensures consistent behavior across different data types and network protocols.

cs/remote/src/FASTER.server/Providers · high confidence

The storage device layer has been restructured to introduce a new \StorageDeviceBase\ abstract class that implements the \IDevice\ interface, providing common functionality for segment management, sector sizes, and file naming (including an option to omit segment IDs from filenames). This change replaces the older \ISegmentedDevice\ interface and its implementations (\SegmentedLocalStorageDevice\, \MemoryDevice\) with a more unified and extensible design. New device types have been added, including \LocalMemoryDevice\ for in-memory storage with configurable latency simulation, \ShardedStorageDevice\ for distributing data across multiple devices using partition schemes, and \TieredStorageDevice\ for composing devices into hot/cold storage tiers with a defined commit point. Additionally, a \ManagedLocalStorageDevice\ using .NET streams and an \AsyncPool\ for efficient handle management have been introduced, alongside a \Devices\ factory class for simplified device creation.

cs/src/core/Device · high confidence

The \Resources.resx\ and \Resources.Designer.cs\ files in the core properties have been removed, as they no longer contained any resources. Additionally, the assembly information has been updated to reflect a 2018 copyright year, and the \InternalsVisibleTo\ attributes for \FASTER.test\ and \FASTER.stress\ have been modified to include a public key, enabling signed friend assembly access.

cs/src/core/Properties · high confidence

The SumStore sample has been updated to use the new session-based FASTER API, replacing the previous static factory and unsafe pointer-based calls with explicit ClientSession management (NewSession, ResumeSession, Dispose). A new RecoveryTest class and updated Program entry point now support concurrency testing and detailed recovery scenarios, including continuing a session after a crash, recovering to the latest checkpoint, or recovering to specific index and hybrid-log checkpoint tokens.

cs/playground/SumStore · high confidence

The SumStore project now includes a launchSettings.json file that configures the default run profile to execute the application with the arguments 'concurrency\_test 2'. Additionally, the AssemblyInfo.cs file has been moved to the project's Properties folder, and several assembly metadata attributes (Title, Configuration, Company, Product, Version, and FileVersion) have been commented out, likely to rely on implicit defaults or simplified project configurations.

cs/playground/SumStore/Properties · high confidence

Added remote test suite for FASTER pub-sub and binary operations; Added unit tests for Azure Blob storage integration; Expanded test coverage for FASTER and F2 key-value stores; Expanded test coverage for FASTER core operations; Initial BenchmarkDotNet performance tests added.

The C\# samples, playgrounds, benchmarks, and core libraries have been migrated from the legacy .NET Framework 4.6 to .NET 7 (with .NET 6 and .NET Standard 2.0/2.1 retained for specific libraries). This change updates all project files to the modern SDK-style format, enabling cross-platform builds on AnyCPU and x64, and upgrades key dependencies such as CommandLineParser to 2.9.1, MessagePack to 3.1.8, and Microsoft.Extensions.Logging to 8.0.1.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 45 → 49 (+3.5)
• Rubric changed (rubric-2026.08.19 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 38 → 37 (-0.6)
• Architecture 97 → 97 (+0.2)
• Maturity 45 → 49 (+3.8)
• Readiness 39 → 60 (+20.6)
• Security 78 → 67 (-11.0)
• Performance 75 → 75 (+0.0)

• BarePragmaDisable (cs/performance/BenchmarkDotNet/InliningTests.cs)
• BarePragmaDisable (cs/performance/BenchmarkDotNet/LightEpochTests.cs)
• BarePragmaDisable (cs/performance/BenchmarkDotNet/SyncVsAsyncTests.cs)
• BarePragmaDisable (cs/samples/ResizableCacheStore/Program.cs)
• BarePragmaDisable (cs/samples/StoreVarLenTypes/AsciiSumSample.cs)
• BarePragmaDisable (cs/src/core/Allocator/BlittableAllocator.cs)
• BarePragmaDisable (cs/src/core/Allocator/GenericAllocator.cs)
• BarePragmaDisable (cs/src/core/Allocator/VarLenBlittableAllocator.cs)
• BarePragmaDisable (cs/src/core/Index/Common/RecordInfo.cs)
• Bounded contexts not declared
• Build failed
• Concentrated knowledge decay
• Duplicated block (10 lines × 2) (cs/src/core/Allocator/ScanIteratorBase.cs)
• Duplicated block (11 lines × 2) (cs/src/core/Allocator/GenericAllocator.cs)
• Duplicated block (11 lines × 2) (cs/src/core/Index/CheckpointManagement/DeviceLogCommitCheckpointManager.cs)
• Duplicated block (11 lines × 3) (cs/src/core/Index/Synchronization/HybridLogCheckpointTask.cs)
• Duplicated block (12 lines × 2) (cs/samples/HelloWorld/Program.cs)
• Duplicated block (12 lines × 2) (cs/samples/StoreVarLenTypes/MemoryByteSample.cs)
• Duplicated block (12 lines × 3) (cs/src/core/Allocator/BlittableScanIterator.cs)
• Duplicated block (13 lines × 2) (cs/samples/HelloWorld/Program.cs)
• …and 68 more

• ClientSession.ProcessReplies (cognitive 26) (cs/remote/src/FASTER.client/JavascriptClient/ClientSession.js)
• ClientSession.ProcessReplies (cyclomatic 20) (cs/remote/src/FASTER.client/JavascriptClient/ClientSession.js)
• CommentedOutCode (cs/playground/CacheStoreConcurrent/Program.cs)
• CommentedOutCode (cs/src/core/Device/LocalStorageDevice.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/ConcurrentDictionaryBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• DeadPreprocessorBranch (cs/benchmark/FasterSpanByteYcsbBenchmark.cs)
• …and 220 more

• 2 commits — 1 feature/other, 1 fixes

• .github/dependabot.yml — 1 commit
• cs/remote — 1 commit

• fix: Security hardening, .NET 10 build fix, and maintenance notice (#960)
• change: Upgrade MessagePack to 3.1.8 and enable Dependabot version updates (#961)

• Unchanged — 7 containers · 1 contexts · 0 edges

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

microsoft/FASTER was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.