dolthub/dolt

Dolt is a version-controlled SQL database that combines Git-like branching, merging, and history management with a full relational query engine. It provides comprehensive data manipulation capabilities, including import/export across multiple formats, automated CI workflows, and granular branch-level access control. The system supports distributed operations through robust remote storage backends, binary log replication for high availability, and extensible authentication mechanisms.

84 changes.

The project rebranded from Noms to Dolt and established formal governance, while simultaneously migrating core CLI operations and data I/O to the SQL engine. This period saw the introduction of comprehensive table management, merge conflict resolution, and credential systems, alongside significant refactoring of the underlying storage and reference handling infrastructure.

82 changes.

This period focused on migrating the core storage engine to a Prolly-based architecture, introducing adaptive encoding, vector indexes, and unified durable storage. It also established foundational infrastructure for cluster replication, including binary log support, JWT authentication, and comprehensive performance benchmarking suites.

63 changes.

This period focused on introducing a native CI workflow engine and enhancing storage reliability through external sorting and robust remote delivery layers. Significant architectural improvements included centralized configuration management, dynamic memory budgeting, and extensible system table adapters to support integrations like Doltgres.

A new benchmarking tool has been added to the \go/store/nbs/benchmarks\ directory to evaluate the performance of the Noms Block Store (NBS). The suite includes benchmarks for novel and duplicate writes, as well as sequential and hash-ordered reads, against local disk, file-based, and AWS S3/DynamoDB-backed stores. The implementation migrates the AWS integration to the \aws-sdk-go-v2\ library and adapts the benchmark harness to the updated \ChunkStore\ interface, which now uses callbacks for \GetMany\ results and \InsertAddrs\ instead of channels.

go/store/nbs/benchmarks · high confidence

Users can now import from and export to Parquet files using the \dolt sql\ command. This change introduces a new Parquet reader and writer in the \go/libraries/doltcore/table/typed/parquet\ package, enabling the handling of various SQL data types including DECIMAL, DATETIME, TIME, and nested structures (lists/arrays) during conversion to and from the Parquet format.

go/libraries/doltcore/table/typed/parquet · high confidence

New files in the \go/store/nbs/benchmarks/gen\ package provide utilities for generating and chunking benchmark data. The \gen.go\ file introduces functions to create or open data files and build chunk maps by reading random bytes and splitting them into chunks based on a rolling hash. The \rolling\_value\_hasher.go\ file implements the rolling hash logic using the \github.com/silvasur/buzhash\ library to determine chunk boundaries.

go/store/nbs/benchmarks/gen · high confidence

Users can now edit commit messages using an external editor (such as vim or Notepad++) via the \dolt sql\ shell. The new \OpenTempEditor\ utility creates a temporary file with initial content, launches the specified editor command, waits for it to close, and returns the modified text, allowing for more complex message composition than simple inline input.

go/libraries/utils/editor · high confidence

Introduces a new output pager utility that intercepts command output for interactive viewing. Users can now set the \DOLT\_PAGER\ environment variable to specify a custom pager command (including arguments), with automatic fallback to \less\ or \more\ if the specified pager is unavailable or not found. The pager respects the \--no-pager\ flag and automatically disables itself when standard output is not a terminal or when running in test environments, ensuring non-interactive scripts remain unaffected.

go/store/util/outputpager · high confidence

The metrics package now includes a new Histogram type that tracks uint64 data points using a log2-based bucketing strategy. This implementation is designed for concurrent access without locking, trading some precision for performance by using atomic operations for updates. It supports time and byte-specific formatting for human-readable output and includes methods for sampling, summing, and calculating means.

go/store/metrics · high confidence

The HTTP utility package now includes an IsLocalRequest function that determines whether an incoming request originated from a local source. It identifies local requests by checking for loopback IP addresses (IPv4 and IPv6) or Unix socket paths, while explicitly treating requests with common proxy headers (such as X-Forwarded-For) as non-local to prevent spoofing. This capability supports scenarios like skipping authentication for localhost metrics endpoints by reliably distinguishing between direct local connections and proxied external traffic.

go/libraries/utils/http · high confidence

This change introduces new utility packages within the Go store library: \go/store/util/writers\ provides \MaxLineWriter\ (which stops writing and returns an error after a specified number of lines) and \PrefixWriter\ (which prepends a custom string to each line), along with \go/store/util/exit\ which offers a mockable wrapper around \os.Exit\ for easier testing, and \go/store/nbs/benchmarks/cachedrop\ which adds a benchmark tool to clear the system page cache.

go/store/nbs/benchmarks/cachedrop, go/store/util/exit, go/store/util/writers · high confidence

The \go/libraries/doltcore/doltdb\ package now includes an \AGENT.md\ file that provides operational guidance for AI agents interacting with Dolt databases. This documentation covers essential CLI commands (such as \dolt init\, \dolt clone\, and \dolt commit\), branch management strategies, and specific behaviors for connecting to the Dolt SQL Server. It also details the usage of the \dolt\_test\ system table for creating and running unit tests, including how to define assertions and interpret test results.

go/libraries/doltcore/doltdb · high confidence

A new command-line utility has been added to analyze the storage efficiency of Noms Block Store (NBS) instances. The tool, located in go/store/nbs/frag, accepts either a local directory or AWS S3/DynamoDB credentials to connect to a store. It traverses the data graph from the root, calculating metrics such as node height, branching factor, and the ratio of actual reads to optimal reads, helping users understand data locality and storage overhead.

go/store/nbs/frag · high confidence

New SQL scripts have been added to the import benchmarker reporting module to facilitate performance analysis. These include \select\_star.sql\ for raw data retrieval, \summary.sql\ for calculating average performance multipliers, and \three\_way\_compare.sql\ for detailed comparisons between MySQL, Dolt CLI, and Dolt Server execution times.

_go/performance/import\benchmarker/reporting · high confidence

New SQL scripts (systab.sql and systab\_summary.sql) have been added to the sysbench reporting module to generate summary reports from sysbench test results. These queries calculate mean performance multipliers and standard deviations by comparing current test runs against previous baselines, enabling users to analyze performance trends and statistical significance in benchmark data.

go/performance/sysbench/reporting · high confidence

The osutil package now includes platform-specific implementations for Windows, introducing functions to handle Windows-native path conversion (PathToNative) and detection of Windows sharing violations (IsWindowsSharingViolation). On Windows, the library initializes system volume and file system root variables, with a fallback to 'C:' if the SYSTEMROOT environment variable is missing, ensuring correct path resolution and error handling on that platform.

go/libraries/utils/osutil · high confidence

This change introduces a new benchmarking tool for the NBS chunker located at go/store/nbs/benchmarks/chunker, which allows users to generate test data or chunk existing files to measure performance. Additionally, a new profiling utility package (go/store/util/profile) has been added, providing flags and helpers to easily enable CPU, memory, and block profiling for Go applications, simplifying the process of gathering performance metrics during execution.

go/store/nbs/benchmarks/chunker, go/store/util/profile · high confidence

A new thread-safe generic map implementation is now available in the concurrentmap package, providing methods for setting, getting, deleting, and iterating over key-value pairs. It includes a DeepCopy method for creating independent copies and custom JSON marshalling/unmarshalling to ensure correct serialization behavior. Comprehensive tests have been added to verify functionality under concurrent access.

go/libraries/utils/concurrentmap · high confidence

A new utility script has been added to the codebase to help reproduce concurrency issues. This tool opens a connection to a local Dolt database and executes a predefined set of SQL statements concurrently across multiple clients and iterations, allowing developers to stress-test the system under concurrent load.

go/utils/concurrency · high confidence

A new \valctx\ package has been introduced to provide a mechanism for validating context lifecycles. This utility allows the application to globally enable context validation, attach validation functions to specific contexts, and trigger those validations, ensuring that context-related invariants are checked when enabled.

go/libraries/utils/valctx · high confidence

A new MutexMap implementation has been added to the session state layer to manage per-table auto-increment locking. This component allows concurrent operations to lock specific table keys while ensuring that waiters can respond to context cancellation, preventing potential deadlocks and allowing queries to be interrupted cleanly.

go/libraries/doltcore/sqle/dsess/mutexmap · high confidence

The go/store/d package now includes try.go, which provides a set of debug and assertion functions (such as Panic, PanicIfError, PanicIfTrue, and PanicIfFalse) that panic when invariants are not met, along with a corresponding test suite in try\_test.go to verify this behavior.

go/store/d · high confidence

Users can now run escape analysis on Go packages using new helper scripts: a .bat file for Windows and a .sh script for Unix-based systems. These tools compile the specified package with flags that disable optimizations and inlining to log escape information, then filter out non-escaping variables and sort the results, providing a convenient way to inspect memory allocation behavior directly from the command line or integrated development environments.

go/utils/escape-analysis · high confidence

Introduced a new generic circular buffer implementation (\Buff\[T\]\) in the \go/libraries/utils/circular\ package, providing fixed-size and auto-growing queue capabilities with \Push\, \Pop\, \Front\, and \At\ operations, along with comprehensive unit tests covering size limits, growth behavior, and error conditions.

go/libraries/utils/circular · high confidence

The import tester utility now includes scripts to generate synthetic CSV data and benchmark the performance of \dolt table import\ and \export\ commands. This allows users to compare the current version of Dolt against a previous release (v0.34.5) to measure improvements in bulk data loading and exporting.

_go/performance/import\tester · high confidence

A new shell script at go/utils/prepr/prepr.sh has been added to help developers verify the repository state locally before submitting a pull request. This utility runs essential checks including building all Go packages, verifying code formatting, checking dependencies, running go vet, and validating copyright headers, mirroring the logic found in the CI workflow to catch common issues early.

go/utils/prepr · high confidence

New scripts have been added to standardize code formatting and test naming conventions. The \check\_fmt.sh\ and \format\_repo.sh\ scripts now enforce Go import ordering using \goimports\ (configured for \github.com/dolthub/dolt\ and \github.com/dolthub/eventsapi\_schema\) and limit Go files to three import groups. Additionally, \check\_bats\_fmt.sh\ ensures that BATS test titles in \integration-tests/bats\ follow the required \"FILENAME: test name"\ format.

go/utils/repofmt · high confidence

The \valutil\ package now includes the \NilSafeEqCheck\ function, which allows users to compare two \types.Value\ instances while treating \nil\ and \types.NullValue\ as equal. This utility simplifies value equality checks by handling null semantics automatically, and is accompanied by unit tests verifying its behavior across various null and non-null scenarios.

go/libraries/utils/valutil · high confidence

Introduced a new \version\ package in \go/libraries/utils/version\ that provides \Encode\ and \Decode\ functions to convert semantic version strings (format X.Y.Z) into and from a compact \uint32\ representation. This allows for efficient numerical comparison of versions, where the major version occupies the highest 8 bits, the minor version the next 8 bits, and the build number the lowest 16 bits. The implementation includes validation to ensure version components fit within these bit constraints and rejects malformed inputs.

go/libraries/utils/version · high confidence

The \3pdeps\ utility now automatically aggregates and outputs the license texts for Dolt, the Go standard library, and statically linked dependencies (musl, mimalloc, ICU4C) along with other Go modules. It also introduces a \-verify\ mode that compares the generated license output against a stored file, enabling CI checks to detect unauthorized or missing license inclusions.

go/utils/3pdeps · high confidence

The project now includes a new \go/Godeps\ directory containing scripts (\update.sh\ and \verify.sh\) and a generated \LICENSES\ file to manage third-party dependencies. The \update.sh\ script aggregates license information for all dependencies across multiple operating systems and architectures (Linux, Windows, macOS) into a single \LICENSES\ file, while \verify.sh\ ensures the file remains up-to-date. This provides a centralized, automated way to track and distribute the licenses of all third-party software used in the product.

go/Godeps · high confidence

The blobstore package now includes an Azure Blob Storage implementation (AzureBlobstore) that supports the standard Blobstore interface, including ranged reads, streaming block uploads, and check-and-put manifest operations. This allows users to store and retrieve data using Azure Blob Storage as a backend.

go/store/blobstore · high confidence

Added README and AWS-specific documentation for the Noms Block Store (NBS) in go/store/nbs. The new files explain how to configure NBS to use Amazon S3 for immutable chunk data and DynamoDB for mutable state, including IAM permission requirements and code examples for instantiating the store.

go/store/nbs · high confidence

The Dolt CI system now supports two distinct step types in workflow configurations: Saved Query steps, which execute a saved query and validate expected row and column counts, and Dolt Test steps, which run tests by group or by individual test name. This change introduces the necessary database schema (including new tables for workflow steps, saved query results, and Dolt test groups/tests), a YAML configuration parser that handles polymorphic step definitions, and a workflow manager to persist and retrieve these configurations.

_go/libraries/doltcore/env/actions/dolt\ci · high confidence

The dolt\_builder utility has been enhanced to allow consumers to supply custom Go build tags via the new RunWithBuildTags API and the -tags CLI flag, enabling the construction of binaries with specific feature sets. Additionally, the tool now respects the DOLT\_REPOSITORY\_URL environment variable to clone source from arbitrary Git repositories instead of the hardcoded default, and supports PGO profiling via the -profile flag for single-revision builds.

_go/performance/utils/dolt\builder · high confidence

The \adapters\ package now exposes a \TableAdapter\ interface and a global registry (\DoltTableAdapterRegistry\) that allows external integrations (such as Doltgres) to customize, wrap, or rename Dolt system tables without modifying core Dolt code. Integrators can register adapters to intercept system table creation, apply type conversions or schema modifications, and define internal aliases (e.g., mapping 'status' to 'dolt\_status') to support cleaner, Postgres-style naming conventions.

go/libraries/doltcore/sqle/adapters · high confidence

The \go/gen/fb/serial\ package now includes generated Go code for a comprehensive set of FlatBuffers message types, including AddressMap, Blob, BranchControl, Collation, Commit, CommitClosure, Encoding, FileIdentifiers, ForeignKey, MergeArtifacts, and ProllyTreeNode. This generated code provides the serialization and deserialization structures required for the database's internal storage format, supporting features such as branch control permissions, foreign key constraints, merge conflict artifacts, and adaptive encoding for data types.

go/gen/fb · high confidence

This change adds the generated Go code for the replication API v1alpha1, introducing new message types and RPC stubs to support replicating user/grant updates and branch control permissions. Specifically, the diff defines \UpdateUsersAndGrantsRequest/Response\ and \UpdateBranchControlRequest/Response\ messages, which carry serialized contents for these respective replication operations, enabling the underlying SQL engine to communicate these permission changes to standby replicas.

go/gen/proto/dolt/services/replicationapi · high confidence

This change introduces the foundational components for binary log replication in Dolt, including the primary controller for serving binlog events, a consumer for applying them on replicas, and utilities for JSON serialization and file management. It adds persistence logic for replication metadata and position tracking, hooks for database lifecycle events, and a log manager for handling binlog file rotation and GTID tracking.

go/libraries/doltcore/sqle/binlogreplication · high confidence

Added a new benchmarking tool in go/performance/kvbench that provides a unified interface for comparing key-value store implementations. The suite includes a memory-based store and a BoltDB-backed store, along with a stub for a Bitcask store. The benchmark framework supports point-read performance testing and includes optional CPU profiling generation via pprof.

go/performance/kvbench · high confidence

The store configuration system now supports a \.nomsconfig\ file that allows users to define database aliases and default database URLs. A new \Resolver\ component reads this configuration to automatically resolve shorthand database names (aliases) and the empty string (default database) into full database specs, simplifying command-line arguments and internal database lookups.

go/store/config · high confidence

The \go/store/hash\ package now provides a dedicated \Hash\ type that represents values using the first 20 bytes of SHA-512, serialized as 32-character base32 strings using a custom Dolt alphabet (0-9, a-v). This change introduces new \base32.go\ and \base32\_test.go\ files to handle the custom encoding and decoding, alongside \hash.go\ which defines the \Hash\ struct, its comparison methods, and parsing logic. The package also includes \HashSlice\ for sortable collections of hashes and comprehensive tests covering parsing, equality, and sorting behaviors.

go/store/hash · high confidence

The \go/libraries/doltcore/creds\ package now provides a new credential system using Ed25519 key pairs instead of the previous implementation. Users benefit from secure, file-based storage of credentials in JWK format (with restricted 0600 permissions) and the ability to generate signed JWTs for outbound RPCs. The \RPCCreds\ type handles token generation with specific audience, issuer, and subject claims, enabling authenticated communication with DoltHub services.

go/libraries/doltcore/creds · high confidence

Added a new XLSX reader implementation in the untyped table package that enables importing Excel files into Dolt. The reader now supports opening XLSX files from both file paths and binary content (via OpenXLSXReaderFromBinary), which facilitates integration with APIs like Dolthub that provide data in memory. The import process validates that the Dolt table name matches the Excel sheet name and verifies schema compatibility before decoding rows, ensuring data integrity during the import operation.

go/libraries/doltcore/table/untyped/xlsx · high confidence

Users can now view and resolve merge conflicts directly via the CLI. The new \dolt conflicts cat\ command displays both schema and data conflicts for specified tables (or all tables using \.\) by querying the underlying conflict system tables and formatting the output as a tabular diff. The \dolt conflicts resolve\ command allows users to automatically resolve conflicts for one or more tables by choosing either the \--ours\ or \--theirs\ version, leveraging SQL-based resolution logic to update the working set.

go/cmd/dolt/commands/cnfcmds · high confidence

Users can now manage Dolt repository documentation (such as README, LICENSE, and AGENT files) directly via the command line. This change adds a new \dolt docs\ subcommand group containing \dolt docs print\ to display the current content of a doc, \dolt docs upload\ to update a doc from a local file, and \dolt docs diff\ to show differences between the working directory and the HEAD version of a doc. These commands interact with the \dolt\_docs\ system table, allowing users to version and maintain standard repository metadata files alongside their data.

go/cmd/dolt/commands/docscmds · high confidence

The \go/store/val\ package now supports adaptive encoding for string, bytes, JSON, geometry, and extended types. This allows the storage engine to automatically decide whether to store large values inline within the row or out-of-band (in separate chunks) based on a configurable target row size, optimizing storage efficiency and access patterns for tables with variable-length data.

go/store/val · high confidence

Added the batsee utility in go/utils/batsee to run BATS integration tests concurrently, significantly reducing execution time compared to serial runs. The tool allows users to configure parallelism via the --threads flag, limit execution duration with --max-time, filter specific tests using --only, skip known slow tests with --skip-slow, and control retry behavior with --retries. Test output is written to a configurable directory (defaulting to batsee\_results), and the utility is restricted to running from the integration-tests/bats directory to ensure correct context.

go/utils/batsee · high confidence

The benchmark runner utility now supports running sysbench and TPC-C workloads against Dolt, Doltgres, MySQL, and Postgres servers. Users can configure benchmark suites via JSON, specifying server connection details, test types (sysbench or TPC-C), and output formats (CSV or JSON). The runner handles server initialization, test execution, and result collection, with specific implementations for each database engine to manage their unique connection protocols and setup requirements.

_go/performance/utils/benchmark\runner · high confidence

This change adds the \dolt ci\ command group to the Dolt CLI, providing a complete set of tools for managing continuous integration workflows directly within the database. Users can now initialize CI storage with \dolt ci init\, import and export workflow definitions via YAML files using \dolt ci import\ and \dolt ci export\, and list or remove workflows with \dolt ci ls\ and \dolt ci remove\. The \dolt ci run\ command executes workflow jobs, supporting both saved query steps and Dolt test steps (with wildcard and intersection selection logic), while \dolt ci view\ displays workflow details and previews the SQL statements that will be executed. The \dolt ci destroy\ command allows for the removal of all CI configuration tables.

go/cmd/dolt/commands/ci · high confidence

The \go/store/skip\ package now provides an in-memory skip list implementation that supports copy-on-update semantics, allowing efficient iteration and retrieval. Users can record checkpoints and revert the list to a previous state using \Checkpoint()\ and \Revert()\, and the list can be cleared entirely with \Truncate()\. Key comparison logic now accepts a \context.Context\ parameter, enabling cancellation and timeout support during lookups and insertions. The implementation includes optimized traversal paths and benchmarks for Get, Put, and Iteration operations.

go/store/skip · high confidence

The \go/store/chunks\ package now provides the core storage abstractions for the Noms data format, including the \Chunk\ struct for representing data units, the \ChunkStore\ interface defining operations like \Get\, \Put\, and \Commit\, and a \MemoryStoreView\ implementation for testing. This change also introduces a \CSMetricWrapper\ to aggregate metrics on chunk operations, a \TableFileStore\ interface for managing table files, and a \MissingChunkError\ for handling missing data scenarios.

go/store/chunks · high confidence

The \go/libraries/doltcore/remotestorage/internal/reliable\ package now provides a new internal infrastructure for guaranteed delivery and retry logic in remote storage operations. For gRPC, it implements a state-machine-driven streaming call wrapper (\grpc.go\, \grpc\_state\_machine\_struct.go\) that handles request/response ordering, automatic retries with backoff, and configurable timeouts for reading requests and delivering responses. For HTTP, it introduces a \StreamingRangeDownload\ mechanism (\http.go\) that streams large byte-range downloads in chunks, automatically retrying transient failures and enforcing minimum throughput thresholds. Both implementations include a generic channel transformer (\chan.go\) to manage unacknowledged message redelivery and a \TimeoutController\ to manage dynamic timeouts across concurrent operations, ensuring robust and reliable data transfer for Dolt's remote storage backend.

go/libraries/doltcore/remotestorage/internal/reliable · high confidence

The \go/utils/remotesrv\ package now provides a standalone server that implements the gRPC remote chunkstore API and a simple HTTP file storage server, allowing Dolt repositories to be hosted remotely. Users can run this server to serve a local directory as a Dolt remote, supporting both standalone mode (creating new repositories) and repo-mode (serving an existing Dolt directory). The server exposes configurable gRPC and HTTP ports, supports read-only mode, and handles graceful shutdown via signal handling.

go/utils/remotesrv · high confidence

The \errhand\ package now provides a new \DErrorBuilder\ and \VerboseError\ interface, allowing CLI commands to present structured error messages that include a concise display message, detailed context, and underlying causes. Users will see clearer, multi-line error output that distinguishes between the primary error and its root cause, with an option to automatically print usage instructions when appropriate.

go/cmd/dolt/errhand · high confidence

A new \dynassert\ utility has been added to provide software integrity sanity checks that are enabled by default during unit and integration tests but disabled in standard production runs. These checks can be manually enabled in controlled environments by setting the \DOLT\_ENABLE\_DYNAMIC\_ASSERTS\ environment variable, allowing the system to fail hard on internal violations while ensuring graceful recovery.

go/libraries/utils/dynassert · high confidence

This change introduces a new command-line argument parsing library in go/libraries/utils/argparser, providing a structured way to define and parse CLI options for Dolt commands. The package includes core components such as the ArgParser struct, Option definitions, and ArgParseResults, along with helper methods for supporting flags, strings, repeatable arguments, and validated inputs. It also includes comprehensive test coverage in args\_test.go and parser\_test.go to ensure correct parsing behavior, including edge cases like empty values, duplicate arguments, and help flags.

go/libraries/utils/argparser · high confidence

The \go/libraries/utils/filesys\ package now provides a unified \Filesys\ interface that abstracts file system operations, allowing the application to work with both the local disk (\LocalFS\) and an in-memory implementation (\InMemFS\) interchangeably. This change introduces a standard set of capabilities including reading and writing files with specific permissions, atomic and durable writes, directory iteration, and file/directory movement. It also includes a \FilesysLock\ interface backed by the \fslock\ library for local file locking and an in-memory lock for testing, along with utility functions for copying files and directories. This abstraction enables consistent file handling across different storage backends and improves testability through the in-memory implementation.

go/libraries/utils/filesys · high confidence

A new \status\ package has been added to \go/store/util/status\ to handle console output. This package provides functions (\Printf\, \Clear\, \Done\) that print status messages to the console, overwriting previous values to create a dynamic progress display. The implementation uses a mutex to ensure thread safety and includes a rate-limiting mechanism to prevent excessive screen updates.

go/store/util/status · high confidence

This change introduces two new capabilities to the JSON table format. First, it adds a JSON diff writer that allows \dolt diff\ to output changes in JSON format, structuring the output with \from\_row\ and \to\_row\ markers for removed, added, and modified rows, along with a separate schema diff section. Second, it adds support for JSON Lines (JSONL) format, enabling both import and export of newline-delimited JSON files. The JSONL reader handles BOM stripping and validates that each line is a valid JSON object, while the JSONL writer outputs each row as a separate line. These changes enhance interoperability with tools that prefer line-delimited JSON and provide structured JSON output for database diffs.

go/libraries/doltcore/table/typed/json · high confidence

The \go/libraries/utils/jwtauth\ package now provides a complete JWT validation framework, introducing a \Claims\ struct that includes an \on\_behalf\_of\ field and a \ValidateJWT\ function that enforces RS256, RS512, and EdDSA algorithms. A new \MultiJWKS\ component allows the system to aggregate and periodically refresh JSON Web Key Sets from multiple URLs, supporting distributed key management. Additionally, the \JWTValidator\ implementation now validates the JWT Subject claim against expected values, ensuring stricter identity verification for authentication flows.

go/libraries/utils/jwtauth · high confidence

Added a new Dockerfile (DataDumpLoadDockerfile) for the integration-tests directory that provisions an Ubuntu 22.04 environment with Go 1.26.2, MySQL client/server, and necessary build dependencies (build-essential, libicu-dev) to run data-dump-loading-tests. This enables CI to build and execute specific data dump loading scenarios using the configured Go and MySQL versions.

integration-tests · high confidence

This change introduces two new Docker images: a CLI image (docker/Dockerfile) and a SQL Server image (docker/serverDockerfile). The SQL Server image allows users to override the root superuser's host and password via the DOLT\_ROOT\_HOST and DOLT\_ROOT\_PASSWORD environment variables, enabling remote connections that were previously restricted to localhost by default. It also supports creating a default database and user via DOLT\_DATABASE, DOLT\_USER, and DOLT\_PASSWORD variables, and accepts server configuration through YAML files in /etc/dolt/servercfg.d/. Both images use tini as the init system and install ca-certificates.

docker · high confidence

The \dolt admin\ CLI subcommand now includes several new tools for database storage inspection, debugging, and recovery. Users can inspect archive files with \dolt admin archive-inspect\ (supporting memory-mapped reading and object-level debugging), examine journal files with \dolt admin journal-inspect\ (including CRC/snappy scanning and root/chunk filtering), and view storage metadata with \dolt admin storage\. Additional recovery utilities include \dolt admin conjoin\ to merge oldgen storage files, \dolt admin newgen-to-oldgen\ to promote new generation files, \dolt admin set-ref\ to manually update reference hashes, and \dolt admin show-root\ to list all dataset root entries.

go/cmd/dolt/commands/admin · high confidence

The \go/libraries/utils/gpg\ package now provides core functionality for GPG integration, including importing keys, listing keys, and checking for key existence. It introduces a \Signer\ type that produces detached GPG signatures for commit payloads and a \Verify\ function to validate those signatures. Additionally, a \DecodeAllPEMBlocks\ helper is included to parse PEM-formatted data, supporting the handling of PGP signed messages.

go/libraries/utils/gpg · high confidence

The SQL engine now supports authenticating users via the \authentication\_dolt\_jwt\ plugin, which validates JSON Web Tokens against a configured JWKS server and enforces expected claims (issuer, audience, subject) from the user's identity string. Additionally, the interactive SQL shell and result output have been refactored into a new printing layer that supports multiple formats (tabular, CSV, JSON, JSON Lines, Parquet, vertical) and displays accurate query timing and row counts in interactive mode.

go/cmd/dolt/commands/engine · high confidence

The \kvexec\ package now includes a new query execution builder that can replace standard row-based operators with optimized KV-side implementations for specific query patterns. This introduces a KV-side lookup join iterator for simple equality joins (including left joins and keyless tables), a KV-side merge join iterator for sorted inputs, and a specialized iterator for \COUNT\ aggregations on single columns. These changes allow Dolt to execute these common operations directly against the Prolly tree storage layer, bypassing the row execution engine for improved performance.

go/libraries/doltcore/sqle/kvexec · high confidence

Added a new Docker-based integration test suite under \integration-tests/orm-tests\ to verify compatibility between Dolt and popular ORM libraries. The suite includes smoke tests for Prisma (Node/TypeScript), TypeORM (Node/TypeScript), MikroORM (Node/TypeScript), and Hibernate (Java), along with a Bats test runner (\orm-tests.bats\) and a Dockerfile that provisions the necessary runtime environments (Node.js, Python, Java/Maven) and tools. This allows users and developers to validate that their ORM applications can perform basic CRUD operations against Dolt.

integration-tests/orm-tests · high confidence

The SQL engine now uses a new set of writer components (prollyTableWriter, prollyWriteSession, prollyIndexWriter) to handle data modifications for tables stored in the new Prolly format. This implementation supports both keyless and key-based tables, managing primary and secondary indexes, auto-increment sequences, and foreign key constraints. It introduces a write session that coordinates table edits and flushes changes to the working set, enabling features like adaptive row encoding and efficient index updates for the new storage backend.

go/libraries/doltcore/sqle/writer · high confidence

The \sqlfmt\ package now includes \row\_fmt.go\ and \schema\_fmt.go\, providing functions to generate SQL statements for data and schema changes. \row\_fmt.go\ adds helpers to format rows into INSERT, UPDATE, and DELETE statements, correctly omitting generated columns from the column list and tuple values. \schema\_fmt.go\ introduces \GenerateDataDiffStatement\ to convert row diffs into SQL, and \GenerateSqlPatchSchemaStatements\ to produce DDL statements (ALTER TABLE, DROP TABLE, etc.) for schema changes, including column additions/removals, renames, type modifications, primary key changes, index updates, foreign key adjustments, and charset/collation or target row size changes.

go/libraries/doltcore/sqle/sqlfmt · high confidence

This change introduces several new SQL functions in the \dfunctions\ package to expose internal Dolt state to users. \active\_branch()\ returns the name of the current branch (or NULL in detached HEAD state), while \dolt\_merge\_base()\ and \has\_ancestor()\ allow querying commit graph relationships. Database and table integrity can be checked using \dolt\_hashof\_db()\ (returning the root hash for a specific ref like WORKING, STAGED, or HEAD) and \dolt\_hashof\_table()\. Additionally, \dolt\_storage\_format()\ reports the current storage format, \dolt\_version()\ returns the server version, and \dolt\_join\_cost()\ prints the query planner's join memo tree for debugging. The \init.go\ file registers these functions, with a separate \DolthubApiFunctions\ list exposing a subset to the Dolthub API.

go/libraries/doltcore/sqle/dfunctions · high confidence

A new command-line tool has been added to run SQL logic tests and parse their results. It supports two modes: 'run' to execute tests against the Dolt harness, and 'parse' to process existing result files. The parser outputs results in either JSON or CSV format (selectable via the -r flag) and includes test duration in milliseconds, allowing users to import test outcomes into Dolt for tracking.

go/libraries/doltcore/sqle/logictest/main · high confidence

A new Dockerfile and supporting configuration files have been added to the continuous integration directory to build a dedicated container for running the benchmark runner tests. This image installs and configures MySQL, PostgreSQL, Dolt, and Doltgres, along with the necessary benchmarking tools (sysbench, TPCC scripts), and executes the Go-based benchmark runner tests via a custom entrypoint script.

_go/performance/continuous\integration · high confidence

A new hidden admin command, \dolt admin createchunk commit\, has been added to allow low-level creation of commit chunks directly at the filesystem level. This command requires a repository context and accepts arguments for the branch name, root hash, parent commits, author details, and a description. It enables administrators to manually construct commit objects and update branch references without relying on the previous SQL-based system procedure, ensuring that unauthenticated SQL users cannot manipulate branches while preserving CLI-level admin capabilities.

go/cmd/dolt/commands/admin/createchunk · high confidence

Users can now run \dolt constraints verify\ to validate that uncommitted changes in the working set satisfy defined table constraints. The command checks for violations (such as foreign key or unique key breaches) against the current working root and records any findings in the \dolt\_constraint\violations\\*\ system tables. It supports an \--all\ flag to verify against an empty root (checking all rows) and an \--output-only\ flag to report violations without persisting them. If violations are found, the command prints a summary and directs users to the specific violation tables for details, returning a non-zero exit code to signal failure.

go/cmd/dolt/commands/cvcmds · high confidence

The \go/store/prolly/tree\ package now includes a new \AddrDiff\ stream and \ChunkAddressDiffOrderedTrees\ function that compare two prolly trees by yielding pairs of node addresses (hashes) for modified nodes rather than full content. This allows callers to identify which specific chunks differ between tree versions without loading the entire node payloads, optimizing the diffing process for large datasets.

go/store/prolly/tree · high confidence

This release introduces the DOLT\_BRANCH\_STATUS table function to show commits ahead/behind for branches, and the DOLT\_JSON\_DIFF table function to compare two JSON objects and return the specific paths and values that differ. The DOLT\_LOG table function now supports bind variables in prepared statements by deferring argument parsing until execution, resolving previous limitations with dynamic queries. Additionally, the DOLT\_PREVIEW\_MERGE\_CONFLICTS table function is added to allow users to inspect merge conflicts before committing them, and existing diff-related functions (DOLT\_DIFF, DOLT\_DIFF\_STAT, DOLT\_DIFF\_SUMMARY) have been updated with better schema handling and authorization checks.

go/libraries/doltcore/sqle/dtablefunctions · high confidence

The \go/libraries/utils/async\ package now includes several new components to support asynchronous workloads: \ActionExecutor\ for managing concurrent task execution with dynamic goroutine scaling and optional buffer limits; \AsyncReader\ for streaming data reads in the background; \RingBuffer\ for thread-safe, dynamically sized producer-consumer queues with epoch tracking; and \GoWithCancel\ for integrating context cancellation with \errgroup\ without triggering group errors on benign cancellations. These additions provide foundational building blocks for improved SQL write performance and safer index lookups.

go/libraries/utils/async · high confidence

The \statspro\ package introduces a new background worker that automatically collects and maintains table statistics for query optimization. This system replaces the previous manual approach by running a dedicated thread that periodically scans databases and indexes to build histogram data, which is then persisted to a dedicated \.dolt/stats\ storage location. The implementation includes a \StatsController\ to manage the worker lifecycle, a \simpleRateLimiter\ to control processing frequency, and a \bucketBuilder\ to aggregate histogram data. Users benefit from more accurate query plans without needing to manually trigger statistics updates, as the system handles collection, persistence, and garbage collection of stats automatically.

go/libraries/doltcore/sqle/statspro · high confidence

The new \go/libraries/doltcore/env/actions/commitwalk\ package provides topological commit graph iterators that return \doltdb\ types, adapting the lightweight \commitgraph\ package to \\*doltdb.DoltDB\. This introduces \GetDotDotRevisions\ and \GetTopologicalOrderIterator\ functions, enabling users to retrieve commits in reverse topological order with specific tie-breaking rules (height, then timestamp) and support for 'dot-dot' revision ranges (e.g., \main..feature\). The implementation includes caching of commit metadata to avoid redundant work and supports iterating across multiple DoltDB instances.

go/libraries/doltcore/env/actions/commitwalk · high confidence

The environment package now includes a credential helper system that follows the Bazel credential helper protocol to obtain and cache authentication headers for remotesapi origins, and a new URL normalizer that translates various git remote formats (including scp-style, schemeless, and local paths) into canonical dbfactory URLs.

go/libraries/doltcore/env · high confidence

Added a new utility script at go/utils/devbuild that allows developers to generate release-like builds for supported platforms (linux-arm64, linux-amd64, darwin-arm64, darwin-amd64, windows-amd64) using Docker with Go version 1.26.2. This tool provides a convenient way to create builds similar to the production release process, though it does not include profile-guided optimization.

go/utils/devbuild · high confidence

Dolt now includes a \dolt creds\ subcommand to manage the public/private keypairs used to authenticate with doltremoteapi. This new CLI group provides \dolt creds new\ to generate a keypair, \dolt creds ls\ to list available keys, \dolt creds use\ to select a specific key for authentication, \dolt creds rm\ to remove a key, \dolt creds import\ to import an existing JWK file, and \dolt creds check\ to verify that a credential can successfully authenticate against the remote API.

go/cmd/dolt/commands/credcmds · high confidence

A new read-only \dolt\_cluster\ database is now available, containing a \dolt\_cluster\_status\ table that reports replication health. The table includes columns for the database name, standby remote, server role (primary or standby), epoch, replication lag in milliseconds, last update timestamp, and any current error messages. This allows users to query the current state of their cluster replication directly via SQL. The implementation also ensures the database correctly implements event database interfaces to prevent spurious error logs from the event scheduler.

go/libraries/doltcore/sqle/clusterdb · high confidence

The remote storage layer now uses a new internal dynamic thread pool (go/libraries/doltcore/remotestorage/internal/pool) that allows runtime scaling of worker threads via SetSize, integrates with context cancellation for clean shutdown, and propagates errors from worker functions through errgroup. This replaces the previous static concurrency model, enabling the system to adjust parallelism for operations like download location retrieval and gRPC communication based on current load and success/failure rates.

go/libraries/doltcore/remotestorage/internal/pool · high confidence

Added two new files to the errors utility package: \panic.go\ introduces a \Fatalf\ function that allows callers to choose between returning an error or crashing the process immediately (with a stack trace) when a fatal error occurs, and \error\_messages.go\ provides a helper to generate user-friendly messages listing databases available to be undropped or indicating none are available.

go/libraries/utils/errors · high confidence

The \expranalysis\ package now provides functions to resolve column default and generated expressions (\ResolveDefaultExpression\), check constraints (\ResolveCheckExpression\), and arbitrary predicates (\ResolveExpression\) by parsing pseudo SQL statements. This allows the system to re-hydrate and validate constraint definitions from their string representations, supporting features like partial indexes and ensuring check definitions match during merges regardless of quoting style. The implementation handles schema-qualified table names and integrates with Doltgres overrides and in-memory parsing contexts.

go/libraries/doltcore/sqle/expranalysis · high confidence

The expreval package now provides a new expression evaluation engine that maps SQL expressions to row-level boolean checks. It introduces a CompareOp interface with implementations for equality, greater/less than, and their inclusive variants, ensuring correct SQL semantics for null handling (e.g., null == null is true, but null \> value is false). The engine supports combining conditions with AND, OR, and NOT logic, and can evaluate comparisons between variables and literals, or variables against a list of literals (IN clauses). This replaces previous ad-hoc comparison logic with a structured, testable approach for filtering rows based on SQL expression criteria.

go/libraries/doltcore/sqle/expreval · high confidence

The rowconv package now includes a new \field\_mapping.go\ file that provides a \NameMapper\ interface and a \NameMapperFromFile\ function to load string-to-string mappings from JSON files, enabling flexible field name translation during data import. Additionally, \row\_converter.go\ introduces a \WarnFunction\ callback type and specific warning constants (such as \DatatypeCoercionFailureWarning\ and \TruncatedOutOfRangeValueWarning\) to allow callers to receive notifications when values cannot be automatically coerced or are truncated, improving visibility into data quality issues during row conversion.

go/libraries/doltcore/rowconv · high confidence

This change introduces two major history-rewriting features to Dolt. First, it adds a \filter-branch\ command (implemented in \filter\_branch.go\) that allows users to rewrite the commit history of one or all branches and tags, including support for applying SQL queries to modify data in each commit and handling uncommitted working/staged changes. Second, it implements the core logic for interactive rebase (in \rebase.go\), introducing a \RebasePlan\ structure with steps for picking, squashing, fixing up, dropping, and editing commits, along with validation logic to ensure plan integrity (e.g., preventing squash/fixup without a preceding pick).

go/libraries/doltcore/rebase · high confidence

The \go/serial\ package now defines the core flatbuffer schemas (\.fbs\) and build infrastructure required to persist Dolt's internal data structures, including \AddressMap\, \ProllyTreeNode\, \Commit\, \RootValue\, \ForeignKey\, and \BranchControl\. This change introduces the serialization format that replaces previous storage methods, enabling efficient encoding of database roots, table schemas, indexes, and merge artifacts.

go/serial · high confidence

The \funcitr\ package now provides functional-style iteration utilities for collections, including \MapStrings\, \FilterStrings\, \MapSlice\, \MapFloat64s\, \MapFloat32s\, \MapInts\, and \MapInt64s\. These functions allow users to map and filter slices of various types while preserving order, with specific handling for nil inputs. The package also includes comprehensive tests covering these operations and nil-slice edge cases.

go/libraries/utils/funcitr · high confidence

A new command-line utility has been added at go/utils/hasher to help developers convert between Dolt base32 hash strings and their raw byte representations. Users can pass a single hash argument to see the underlying bytes, or provide multiple byte values to generate the corresponding base32 hash string, with the remaining bytes padded to zero if fewer than the full length are provided. The utility includes validation for invalid hash formats and handles byte range errors appropriately.

go/utils/hasher · high confidence

Added a new command-line tool in the import benchmarker package to measure import performance. The tool accepts a test file path and an optional output path, runs defined import tests against either an external server, an in-process SQL server, or the CLI, and outputs the results as an SQL dump.

_go/performance/import\benchmarker/cmd · high confidence

The \go/store/blobstore/internal/git\ package introduces a new \GitAPI\ interface and its \GitAPIImpl\ implementation, enabling the blobstore to perform Git plumbing operations (such as resolving refs, reading/writing trees, and managing index files) without requiring a working tree checkout. This abstraction allows the GitBlobstore to execute read and write operations via the Git CLI, supporting features like remote fetch/push with lease semantics and atomic reference updates.

go/store/blobstore/internal · high confidence

Added the \dolt index\ subcommand group, which includes \dolt index ls\ to list indexes (optionally filtered by table), \dolt index cat\ to display index contents in tabular, CSV, or JSON formats, and \dolt index rebuild\ to clear and reconstruct an index's data from the working set. These commands are marked as internal debugging tools intended for developers to inspect index state or temporarily fix out-of-sync indexes.

go/cmd/dolt/commands/indexcmds · high confidence

The iohelp package now includes a CopyFile helper for efficient file-to-file copying, a ReadLine function that correctly handles mixed line endings (\\n and \\r\\n), and a ReaderWithStats wrapper that tracks bytes read and elapsed time to provide progress updates. Additionally, the package expands its write capabilities with WriteAll for complete buffer writes, WriteIfNoErr and WritePrimIfNoErr for error-conditional chaining, and WriteWithoutNewLine for precise output control.

go/libraries/utils/iohelp · high confidence

Added a new \keymutex\ package that provides a \Keymutex\ interface and a \NewMapped\ implementation, allowing callers to acquire exclusive access to critical sections identified by a specific string key. This enables concurrent progress for different keys while serializing access for the same key, with support for context cancellation and automatic cleanup of unused mutex states to keep the internal map clean.

go/libraries/utils/keymutex · high confidence

A new \commitgraph\ package has been added to provide a lightweight topological commit graph iterator that does not depend on the SQL engine (\doltdb\ or \go-mysql-server\). This allows consumers, such as DumboDB, to traverse commit history without pulling in heavy database dependencies. The package exposes interfaces for resolving commit hashes and iterators for walking commits in reverse topological order or computing set differences (similar to \git log excluded..included\), supporting filtering via a match function and handling ghost commits.

go/libraries/doltcore/commitgraph · high confidence

The sql-server command now includes a local credential system that writes a \sql-server.info\ file to the \.dolt\ directory, enabling CLI tools to automatically discover and authenticate with a running server using a shared secret and process ID. Additionally, a new Model Context Protocol (MCP) service can be enabled via \--mcp-port\, providing an HTTP interface for external tools to interact with the database, complete with its own port validation and lifecycle management. The server also introduces a structured log format that includes connection IDs and standardized timestamping for better observability.

go/cmd/dolt/commands/sqlserver · high confidence

Added new shell scripts (local\_sysbench.sh, local\_tpcc.sh, local\_custom.sh) and associated Lua benchmarks to the performance suite, enabling local execution of sysbench and TPC-C workloads against a Dolt server instance. These scripts automatically configure a local server with SSL/TLS enabled using test certificates, support pprof CPU profiling via a --pprof flag, and allow customization of table sizes and specific benchmark types. A .gitignore file was also added to exclude cloned Lua script directories.

go/performance/scripts · high confidence

The \go/libraries/doltcore/merge\ package introduces new core merge logic and test coverage. \action.go\ adds a \MergeSpec\ struct and builder pattern to configure merge parameters (e.g., fast-forward modes, squash, force) and an \AbortMerge\ function to safely revert active merges. \fulltext\_rebuild.go\ and \fulltext\_table.go\ implement logic to detect and rebuild Full-Text index pseudo-tables after a merge, ensuring search indexes remain consistent when parent tables are modified on both branches. Additionally, \data\_merge\_test.go\ adds regression tests for data merging, specifically covering edge cases like shifting node boundaries in the storage tree.

go/libraries/doltcore/merge · high confidence

Added a new \minver\ package that enables automatic nulling of struct fields based on version tags. By annotating fields with \minver\ tags (e.g., \minver:"0.0.2"\), the system ensures that fields introduced in later versions are excluded from YAML output when serializing for older versions, supporting backward-compatible data exchange. The package includes validation logic to enforce that tagged fields are nullable and use \omitempty\, along with test coverage for the nulling and validation behavior.

go/libraries/utils/minver · high confidence

A new command-line utility has been added at go/utils/genminver\_validation that generates a minimum version validation file. When executed with an output file path as an argument, it uses the minver package and server configuration to produce the validation data, streamlining the process of creating these files for the project.

_go/utils/genminver\validation · high confidence

The \integration-tests/mysql-client-tests\ directory now contains a comprehensive, Docker-based integration test suite for Dolt's MySQL protocol compatibility. The new \Dockerfile\ builds a unified runtime image containing client drivers and test binaries for C, C++, C\#, Go, Java, Node.js, PHP, Python, Ruby, Rust, Swift, Elixir, Perl, and R. This allows developers to verify that Dolt correctly handles standard SQL operations, prepared statements, and Dolt-specific commands (such as \dolt\_commit\ and \dolt\_merge\) across a wide variety of official and community-maintained database connectors.

integration-tests/mysql-client-tests · high confidence

Added a new \MultiRepoTestSetup\ struct in the \testcommands\ package to facilitate testing scenarios involving multiple Dolt repositories. This utility allows test authors to programmatically create, configure, and manage multiple local database environments and their associated remotes within a shared temporary directory, simplifying the setup for integration tests that require complex repository interactions.

go/libraries/doltcore/dtestutils/testcommands · high confidence

The \go/store/prolly\ package introduces dedicated map implementations for managing address lookups (\AddressMap\), database artifacts like conflicts and constraint violations (\ArtifactMap\), and commit history (\CommitClosure\). These new types provide structured, serializable storage for these specific data domains, backed by a new \MapInterface\ and \MutableMapInterface\ that standardize how prolly-tree maps are accessed, mutated, and merged across the system.

go/store/prolly · high confidence

This change introduces three new shell scripts in the release tooling: \buildindocker.sh\ automates the cross-compilation of Dolt binaries for Linux, macOS, and Windows (including RPM packaging for Linux) using specific toolchains and build tags like \icu\_static\; \buildpgobinaries.sh\ enables building PGO-optimized binaries by wrapping the Docker build script with profile data; and \install.sh\ provides a user-facing installer that downloads the appropriate release artifact and installs the \dolt\ binary to \/usr/local/bin\ on supported Linux and macOS architectures.

go/utils/publishrelease · high confidence

The remotesrv package now exposes a new gRPC-based remote storage API (remotesapi) alongside its existing HTTP file server. This new service introduces role-based access control, requiring SuperUser credentials for write operations (Commit, AddTableFiles) and CLONE\_ADMIN grants for read operations (HasChunks, GetDownloadLocations). To prevent URL forgery, the gRPC server now seals download and upload URLs using AES-256-GCM symmetric encryption with expiration timestamps, which the HTTP handler then unseals before serving files. The server also supports multiplexing HTTP and gRPC traffic on a single port using unencrypted HTTP/2 (h2c) and respects the X-Forwarded-Proto header to generate correct URL schemes when running behind TLS-terminating reverse proxies.

go/libraries/doltcore/remotesrv · high confidence

The sqlutil package now provides core utilities for converting between Dolt's internal schema representation and the SQL engine's schema model. The new convert.go file introduces FromDoltSchema and ToDoltSchema functions to map column definitions, primary key ordinals, and constraints, while ToDoltCol handles individual column details including defaults, generated columns, and auto-increment settings. The schema.go file adds ParseCreateTableStatement to parse DDL and build Dolt schemas with support for indexes, check constraints, and vector properties. Additionally, sql\_row.go introduces SqlColToStr for converting SQL column values to strings (including binary-as-hex and spatial types) and DatabaseTypeNameToSqlType for mapping MySQL wire protocol types. The static\_errors.go file provides StaticErrorTable and StaticErrorEditor for testing error handling.

go/libraries/doltcore/sqle/sqlutil · high confidence

The \dolt schema\ command now includes several new subcommands to help manage schema metadata and exports. Users can now use \dolt schema copy-tags\ to copy column tags from one branch to another, which is useful for resolving merge conflicts related to column tag mismatches. The \dolt schema update-tag\ command allows updating the tag of a specific column to fix schema conflicts. Additionally, \dolt schema export\ enables exporting table schemas as SQL DDL statements, and \dolt schema tags\ displays the column tags for one or more tables. These commands are part of the \schcmds\ package and are designed to provide more granular control over schema operations.

go/cmd/dolt/commands/schcmds · high confidence

A new \sequences\ package has been introduced to define the core interfaces for managing incrementing state (such as AUTO INCREMENT columns) that must be shared across all branches and transactions. The \SequenceState\ interface specifies methods for advancing the sequence, retrieving current values, and merging states from different branches, while the \SequencedRelation\ interface provides the contract for database objects (like tables) that wrap this state. This change establishes the foundational API for ensuring global uniqueness of created rows in a distributed environment.

go/libraries/doltcore/sqle/globalstate/sequences · high confidence

The \go/libraries/utils/set\ package now includes \ByteSet\, \StrSet\, and \Uint64Set\ implementations, each providing standard set operations such as adding, removing, checking containment, and computing intersections. \StrSet\ additionally supports case-insensitive matching and offers methods to convert sets to sorted or unsorted slices, while \Uint64Set\ includes an intersection method and iteration support. These utilities are accompanied by unit tests to verify their correctness.

go/libraries/utils/set · high confidence

The \go/store/util/sizecache\ package introduces a new \SizeCache\ that evicts items based on a specified byte-size limit rather than just count. It uses a \Key\ mechanism to ensure that values fetched before a cache \Purge()\ are not re-inserted after the purge, preventing stale data from repopulating the cache. The implementation includes an optional expiration callback and is designed to be thread-safe.

go/store/util/sizecache · high confidence

The strhelp package now includes new utilities for string manipulation: NthToken extracts a specific delimited token from a string, CommaIfy formats integers with thousand separators, and LineStrBuilder provides a fluent interface for constructing multi-line strings. Tests are included to verify the behavior of these new functions.

go/libraries/utils/strhelp · high confidence

A new \structwalk\ package has been added to \go/libraries/utils\, providing a \Walk\ function that recursively traverses all fields of a Go struct. This utility allows consumers to inspect struct fields along with their reflection types, JSON tags, and nesting depth, supporting nested structs, pointers, and slices. Tests confirm it correctly handles mixed field types and tracks depth for nested structures.

go/libraries/utils/structwalk · high confidence

A new \sysbench\ CLI tool has been added at \go/performance/sysbench/cmd/main.go\ to execute system table performance tests. Users can now run test suites defined in external files using the \-run\ flag, specify configuration and script directories via \-config\ and \-script-dir\, and control output verbosity with \-verbose\. Results are printed to stdout or saved to a file specified by \-out\, supporting both external server and in-process SQL server test execution modes.

go/performance/sysbench/cmd · high confidence

Adds a new Go-based test harness in go/performance/sysbench that allows running sysbench workloads against Dolt servers using YAML-defined test scripts. The new code includes a test definition parser (testdef.go) for configuring database connections and sysbench parameters, a result parser (sysbench\_test.go) that extracts latency histograms and statistics from sysbench output, and documentation (README.md) for local setup. This enables automated performance regression testing for system table operations.

go/performance/sysbench · high confidence

Users now have access to several new SQL system tables in the dolt database schema. The \dolt\_backups\ table exposes configured backup URLs and parameters. The \dolt\blame\\<table\>\ views provide line-by-line commit attribution for table data. The \dolt\_branch\_activity\ table tracks read/write timestamps and active session counts per branch. Additionally, \dolt\_branch\_control\ and \dolt\_branch\_namespace\_control\ tables allow managing branch-level access permissions and namespace restrictions directly via SQL.

go/libraries/doltcore/sqle/dtables · high confidence

The \go/libraries/doltcore/table\ package now exposes a structured framework for row-based table operations, introducing \Reader\, \SqlTableReader\, and \SqlRowWriter\ interfaces to standardize how data is read and written. A new \BadRow\ error type allows callers to handle malformed rows gracefully without immediate failure, and a \RowIter\ wrapper simplifies iteration over SQL rows by abstracting away the underlying SQL context. These changes provide a cleaner, more consistent API for table data access.

go/libraries/doltcore/table · high confidence

The \dolt table\ command now includes four new subcommands for managing table structure and data outside of the import workflow. \dolt table cp\ copies a table (optionally overwriting an existing target with \--force\), \dolt table mv\ renames a table (also supporting \--force\ to overwrite), and \dolt table rm\ removes one or more tables from the working set, with protection against deleting system tables. Additionally, \dolt table export\ allows exporting table contents to files in CSV, PSV, JSON, JSONL, SQL, or Parquet formats, with support for overwriting existing files via \--force\ and explicit format specification via \--file-type\.

go/cmd/dolt/commands/tblcmds · high confidence

The \go/libraries/events\ package introduces a new telemetry framework that collects and reports usage data. Events are gathered by a \Collector\ which batches them and sends them via pluggable \Emitter\ implementations (gRPC, file, stdout, logger, or null). To prevent unbounded memory growth during network interruptions or high event volume, the collector drops oldest events when the queue exceeds the maximum batch size (64 events). Additionally, a \FileFlusher\ component enables offline event persistence by writing event batches to files in the \.dolt/eventsData\ directory, which can be flushed to the telemetry server later using file locking to ensure safe concurrent access.

go/libraries/events · high confidence

This change introduces two new files to the repository. In go/store/util/clienttest, a new ClientTestSuite is added to facilitate testing by capturing stdout and stderr, managing temporary directories, and mocking exit behavior. In go/store/util/progressreader, a new io.Reader wrapper is provided that reports read progress to a callback function at specified intervals.

go/store/util/clienttest · high confidence

The \dtestutils\ package now provides standardized helpers for setting up test environments and schemas. \CreateTestEnv\ and \CreateTestEnvWithName\ allow tests to initialize isolated Dolt environments using either in-memory or local filesystems, while \CreateSchema\ and \MustSchema\ simplify the creation of schema objects from column definitions. Additionally, \AddColumnToSchema\ offers a utility for extending existing schemas, and \data.go\ introduces a pre-configured \people\ test table schema with an index and check constraint for use in integration tests.

go/libraries/doltcore/dtestutils · high confidence

A new TypeCompatibilityChecker has been introduced in the schema package to determine whether column type changes during schema modifications are safe to perform automatically without requiring a full table rewrite. The checker supports specific compatible transitions for string types (such as widening VARCHAR length or moving between inline and out-of-band storage like TEXT), additive changes to ENUM and SET types, and handles collation changes by marking secondary indexes as invalidated. It explicitly rejects incompatible changes, such as narrowing string widths, removing ENUM/SET values, or changing character sets, and ignores custom ExtendedTypes to prevent errors from integrator-defined types.

go/libraries/doltcore/schema/typecompatibility · high confidence

This change introduces dedicated typeinfo implementations for several SQL types that were previously handled differently or lacked specific storage definitions. Specifically, it adds support for the MySQL Bit type (bit.go), a new blobStringType for large TEXT columns (blobstring.go), and a comprehensive set of spatial geometry types including Point, LineString, Polygon, MultiPoint, MultiLineString, MultiPolygon, GeometryCollection, and a generic Geometry type (point.go, linestring.go, polygon.go, multipoint.go, multilinestring.go, multipolygon.go, geometry\_collection.go, geometry.go). It also includes an extendedType bridge for external SQL types (extended.go). These new files define how these types are serialized, compared, and mapped to internal storage encodings, ensuring consistent handling for spatial data and large text/blob fields.

go/libraries/doltcore/schema/typeinfo · high confidence

The \go/libraries/utils/config\ package introduces a new configuration framework that supports hierarchical lookups, file persistence, and thread-safe concurrent access. Users can now organize settings into prioritized layers (via \ConfigHierarchy\) where earlier-added configs take precedence, and access configuration values stored in JSON files (\FileConfig\) with proper locking to prevent race conditions. The system also provides a \PrefixConfig\ decorator for namespacing keys, useful for isolating SQL server global variables, and exposes a standardized set of configuration keys (such as \user.email\, \remotes.default\_host\, and \mmap\_archive\_indexes\) through \keys.go\.

go/libraries/utils/config · high confidence

The \go/libraries/doltcore/table/untyped\ package now provides utilities to work with untyped table data where all fields are treated as strings. Users can generate a schema with \NewUntypedSchema\ (which creates columns of \StringKind\ and designates the first column as the primary key) or convert an existing schema to use string types via \UntypeSchema\. Additionally, \NewRowFromStrings\ allows constructing rows from string slices, facilitating easier handling of data that does not require strict type enforcement.

go/libraries/doltcore/table/untyped · high confidence

Added a new \goroutinedump\ utility package that provides a \DumpGoRoutines\ function. This function captures and returns a string containing the stack traces of all currently running goroutines in the program by leveraging the \runtime/pprof\ package, enabling operational inspection of concurrent execution states.

go/libraries/utils/goroutinedump · high confidence

A new utility function, All, has been added to the store's function utilities package. This function allows users to execute multiple error-returning functions in parallel and waits for all of them to complete, returning the first error encountered if any function fails. This provides a convenient way to run independent tasks concurrently without manual synchronization logic.

go/store/util/functions · high confidence

Dolt cluster replication now synchronizes user and grant changes, as well as branch control permissions, from the primary server to standby replicas. This ensures that authentication and authorization configurations are consistent across the cluster, preventing permission mismatches after a failover. The implementation includes dedicated replication logic for these metadata changes, ensuring they are applied to standby servers alongside standard data replication.

go/libraries/doltcore/sqle/cluster · high confidence

The server configuration files (YAML) now support environment variable placeholders using the ${VAR} syntax, allowing sensitive values like passwords or paths to be injected at runtime. The system also supports escaping literal dollar signs with $$. If a referenced environment variable is unset or empty, the configuration load will fail with an error, ensuring misconfigurations are caught early rather than silently masking issues.

go/libraries/doltcore/servercfg · high confidence

New stored procedures have been added to the SQL interface for \dolt add\, \dolt backup\, \dolt branch\, \dolt checkout\, and \dolt cherry-pick\. These procedures allow users to perform version control operations directly via SQL calls (e.g., \CALL DOLT\_ADD(...)\, \CALL DOLT\_CHECKOUT(...)\), mirroring the functionality of their corresponding CLI commands. The \dolt\_backup\ procedure supports adding, removing, syncing, and restoring backups, while \dolt\_checkout\ handles branch switching and table checks out with support for flags like \--move\ and \--track\. The \dolt\_cherry\_pick\ procedure returns detailed conflict information, including data, schema, and constraint violations.

go/libraries/doltcore/sqle/dprocedures · high confidence

The \go/store/prolly/sort\ package now includes an external sorting implementation (\external.go\) that handles datasets too large to fit in memory. This new component sorts tuples by batching them in memory, spilling sorted batches to temporary disk files, and performing a k-way merge to produce the final sorted output. This capability enables index rebuilds to stream sorted edits efficiently, preventing out-of-memory errors during large-scale index operations.

go/store/prolly/sort · high confidence

The \durable\ package now provides a single, unified interface for persisting Dolt tables, their primary and secondary indexes (including vector/proximity indexes), and merge artifacts (conflicts and constraint violations). This change consolidates previously scattered storage logic into a cohesive \Table\, \Index\, \IndexSet\, and \ArtifactIndex\ model, ensuring that all table metadata and data structures are managed through a consistent, address-based persistence layer.

go/libraries/doltcore/doltdb/durable · high confidence

The \GCSafepointController\ and its associated session lifecycle management have been extracted from the \dsess\ package into a new, SQL-agnostic \gcctx\ package. This change decouples garbage collection safepoint logic from the SQL session implementation, providing a generic context-based API (\WithGCSafepointController\, \SessionCommandBegin\, \SessionCommandEnd\) that allows any component to register for GC safepoint notifications without depending on SQL-specific types.

go/libraries/doltcore/doltdb/gcctx · high confidence

The Dolt CLI has been restructured to use a new \CliContext\ interface for managing command execution state, including global arguments, configuration, and database connections. This change introduces a \LateBindQueryist\ pattern that defers the creation of the SQL engine until it is actually needed, caching the result to avoid redundant connections. Additionally, argument parsing logic has been consolidated into dedicated helper functions (e.g., \CreateCommitArgParser\, \CreateMergeArgParser\) and utility files (\arg\_helpers.go\, \arg\_parser\_helpers.go\), separating parsing concerns from command execution. The \Command\ interface now requires a \CliContext\ parameter in its \Exec\ method, enabling subcommands to access global state and shared resources more effectively.

go/cmd/dolt/cli · high confidence

The repository management logic for branch operations, checkout, cloning, and committing has been consolidated into the new \go/libraries/doltcore/env/actions\ package. This change introduces dedicated files for managing branch creation, deletion, and renaming; handling working set and root transitions during checkout; orchestrating the clone process with improved error handling and cleanup for interrupted operations; and staging commits with constraint and foreign key validation. These refactors centralize the implementation of these core capabilities, improving code organization and maintainability.

go/libraries/doltcore/env/actions · high confidence

The logictest suite now includes \import.sql\ and \regressions.sql\ to support automated regression detection. \import.sql\ defines commands to import CSV data (specifically \headerless\_nightly\_mean.csv\ and \headerless\_releases\_mean.csv\) into Dolt tables, while \regressions.sql\ creates the necessary schema (tables for nightly and release results) and SQL views to identify performance regressions (duration changes \>5%) and result discrepancies between release and nightly builds.

go/libraries/doltcore/sqle/logictest · high confidence

The branch control system has been rewritten to use Flatbuffers for persistence and introduces a new \Permissions\_Merge\ flag. This allows administrators to grant users the ability to merge into a branch without granting full write access. The permission hierarchy now explicitly distinguishes between Admin, Write, Merge, and Read, with higher permissions implicitly including lower ones (e.g., Admin includes Write, Merge, and Read). Existing branch control data is automatically migrated to this new format, where the previous 'Read' bit is remapped to the new 'Merge' bit to preserve intended access levels.

_go/libraries/doltcore/branch\control · high confidence

The CSV reader and writer in \go/libraries/doltcore/table/untyped/csv\ have been rewritten to improve robustness and flexibility. The reader now automatically detects and strips UTF-8, UTF-16LE, and UTF-16BE Byte Order Marks (BOMs) from the start of files. It also supports custom field delimiters (configurable via \CSVFileInfo.Delim\) and correctly handles quoted fields containing commas, newlines, and escaped double quotes. The writer respects these delimiter settings and applies special quoting logic to ensure data integrity, such as properly escaping empty strings and null values.

go/libraries/doltcore/table/untyped/csv · high confidence

The dconfig package now provides a centralized registry of environment variables (such as DOLT\_CLI\_PASSWORD, DOLT\_ROOT\_PATH, and DOLT\_GC\_SAFEPOINT\_CONTROLLER\_CHOICE) and a new date parsing utility. This change standardizes how configuration options are defined and allows for more flexible parsing of date strings in configuration files, supporting multiple common formats like ISO 8601 and US-style dates.

go/libraries/doltcore/dconfig · high confidence

The \go/store/datas\ package now stores a materialized parent closure for each commit, enabling sub-linear \FindCommonAncestor\ calculations and more efficient commit graph traversal during pull, fetch, and push operations. This change introduces a new \parents\_closure\ field in the commit flatbuffer schema and implements the \writeFbCommitParentClosure\ logic to merge parent closures from existing commits. Additionally, the \FindCommonAncestor\ algorithm has been updated to utilize a heap-based approach for improved performance, and the commit structure now supports both the legacy parent Set and the new ordered parent List to maintain backward compatibility.

go/store/datas · high confidence

The \dolt\ CLI command structure has been refactored to use a centralized registration file (\doltcmd.go\). This change consolidates the definition of all subcommands—including SQL, versioning, data manipulation, and administrative tools—into a single \doltSubCommands\ slice. This simplifies the command-line interface's internal wiring and ensures a consistent, unified entry point for all Dolt operations.

go/cmd/dolt/doltcmd · high confidence

The system now uses an atomic write procedure to persist MySQL database privileges (users and grants), ensuring data integrity against crashes during file operations. This change also restricts the file permissions for the privilege storage file to 0600, enhancing security by limiting access to the owner only.

_go/libraries/doltcore/sqle/mysql\_file\handler · high confidence

The database factory in go/libraries/doltcore/dbfactory has been restructured to support a wider range of remote storage backends, including new implementations for Azure Blob Storage (az://), generic S3-compatible stores (s3://), and OCI, alongside the existing AWS (aws://) and Google Cloud (gs://) factories. The AWS factory has been migrated to the aws-sdk-go-v2, and the system now supports SSH remote URLs with explicit port numbers. For local file-based databases, the factory now enforces a singleton cache by default to reuse open database instances, but provides a new disable\_singleton\_cache parameter to bypass this cache for deterministic reopen semantics in embedded drivers. Additionally, git-based remotes (git+file, git+http, etc.) now use a configurable local cache directory (git\_cache\_root) and include a custom gRPC load-balancing policy to improve resilience against endpoint changes behind proxies.

go/libraries/doltcore/dbfactory · high confidence

The internal cache sizes for the node cache, memtable, and decoded chunks are now derived from the Go runtime's GOMEMLIMIT setting. When GOMEMLIMIT is configured, the system allocates 50% of that limit to caches (split 47% node cache, 47% memtable, 6% decoded chunks) while respecting minimum and maximum bounds; if GOMEMLIMIT is not set, the system falls back to the previous static defaults (256 MiB for node cache and memtable, 32 MiB for decoded chunks).

go/libraries/doltcore/memlimit · high confidence

The diff logic in go/libraries/doltcore/diff now supports detecting table renames by matching tables with identical primary key column types rather than relying solely on name equality, and it exposes database-level schema changes (additions and drops of schemas) via the new DatabaseSchemaDelta type. Additionally, the diff subsystem introduces a DiffSplitter to cleanly separate 'from\' and 'to\' columns in diff query results, and provides detailed schema diffing for columns, indexes, and foreign keys, along with improved diff statistics reporting.

go/libraries/doltcore/diff · high confidence

The URL parsing logic in the earl utility has been updated to handle specific edge cases that previously failed or behaved inconsistently. It now supports AWS remote URLs using a non-standard bracketed host syntax (e.g., \aws://\[bucket:table\]/path\), which is required for compatibility with newer Go versions that enforce stricter hostname validation. Additionally, the parser now correctly normalizes Windows file paths, including handling network locations (e.g., \C$/\) and ensuring consistent forward-slash separators, while also fixing issues with file paths starting with a dot.

go/libraries/utils/earl · high confidence

The file utility library now includes a memory-mapped file implementation (mmap.go) that leverages the github.com/edsrzf/mmap-go library to allow mapping specific regions of a file into memory. Additionally, Windows-specific file operations (Rename, Remove, RemoveAll) have been updated to automatically retry upon encountering ACCESS\_DENIED or SHARING\_VIOLATION errors, significantly reducing transient file access failures on Windows. The atomic file writing helper (WriteFileAtomically) has also been refined to ensure the directory inode is synced after renaming, improving data durability.

go/libraries/utils/file · high confidence

Read replicas now retry optimistic lock failures when updating the working set and optimize branch syncing to prevent race conditions when multiple sessions replicate the same head. The system also prevents duplicate database overwrites in the catalog, fixes a deadlock in DROP DATABASE, and ensures that databases created via dolt\_clone() are properly dropped without requiring a server restart.

go/libraries/doltcore/sqle · high confidence

The index creation process in the table editor has been refactored to build secondary indexes using an external sorting mechanism. Instead of materializing all index keys in memory, the new \BuildProllyIndexExternal\ function streams primary key rows, applies optional WHERE predicates (partial indexes), and writes sorted keys to temporary files before materializing the final index. This change improves scalability for large tables by managing memory usage more effectively. Additionally, the new code path explicitly supports building vector (proximity) indexes, which are handled via a specialized \BuildProximityIndex\ function that inserts keys into a proximity map builder.

go/libraries/doltcore/table/editor/creation · high confidence

The \go/store/types\ package now uses a \NomsBinFormat\ type to manage binary serialization formats, replacing the previous hardcoded \Format\_7\_18\ with a new \Format\DOLT\. This change introduces a new \\\_DOLT\DEV\\_\ format for development and enables the storage of \Extended\ types, which are marked as invalid in the old format. The diff also adds support for new spatial geometry types (\Geometry\, \LineString\, \GeometryCollection\) and \InlineBlob\, updating the serialization logic to handle these new value kinds.

go/store/types · high confidence

The verbose logging utility now uses the zap library instead of a generic log function, providing a Logger variable that accepts a context and returns a \*zap.Logger. This change allows callers to override the logging behavior and supports context-aware logging throughout the store layer. Verbosity is controlled via -v/--verbose flags or the SetVerbose function, switching between Debug and Warn levels accordingly.

go/store/util/verbose · high confidence

Dolt now tracks auto-increment values globally rather than per-branch, ensuring that the next auto-increment ID is determined by the highest value across all branches and merges. This change introduces a new \GlobalState\ interface and \SequenceTracker\ system to manage these values, which helps maintain consistent ID sequences during operations like \dolt\_reset --hard\ or when merging branches with divergent auto-increment histories.

go/libraries/doltcore/sqle/globalstate · high confidence

The temporary file system now uses a lazy-loading provider for movable temp files. Instead of immediately validating or setting up the provider at startup, the system defers this work until the first time a temporary file is actually needed. This change helps avoid early startup errors (such as access denied issues on Windows or cross-volume rename failures) by ensuring that the required capabilities are only checked when the functionality is first used, while still maintaining thread-safe access to the underlying file operations.

go/store/util/tempfiles · high confidence

The Dolt CLI now uses a lazy configuration for movable temporary files, deferring the check of whether the system temp directory can be moved to the data directory until it is actually needed, which prevents unnecessary filesystem writes. Additionally, the CLI now includes platform-specific file descriptor checks (on Linux and macOS) that warn users if the available file descriptors are below the recommended threshold, and it ensures the temp directory is correctly configured within the data directory if cross-volume moves are not supported.

go/cmd/dolt · high confidence

The \dolt status\ command has been rewritten to generate its output using SQL queries instead of direct database access. This change aligns the command with the broader migration of CLI operations to the SQL engine, ensuring consistent behavior and leveraging the existing SQL infrastructure for status reporting.

go/cmd/dolt/commands · high confidence

Dolt now uses a new Prolly-based index implementation for both primary and secondary indexes, replacing the previous Noms-based storage engine. This change introduces new index iterators (prollyIndexIter, prollyKeylessIndexIter, prollyCoveringIndexIter) that work with the Prolly map data structure, enabling more efficient index lookups and range scans. The implementation supports keyless tables, covering indexes, and proper handling of virtual columns and prefix indexes through the new SecondaryKeyBuilder. System table indexes (DOLT\_DIFF, DOLT\_BRANCHES) are also migrated to use the new Prolly-based index infrastructure.

go/libraries/doltcore/sqle/index · high confidence

The SQL shell now uses a dedicated prompt resolver to dynamically display the current base database, active revision (branch or commit), and dirty state in the command-line interface. This change introduces a new \resolver.go\ module that chains resolvers to fetch this information via SQL functions like \database()\, \active\_branch()\, and \dolt\_status\, ensuring the prompt accurately reflects the session's context, including handling non-branch revisions and non-Dolt databases gracefully.

go/cmd/dolt/cli/prompt · high confidence

The core cherry-pick logic has been refactored into a dedicated \cherry\_pick\ package, introducing a new \CherryPick\ function that replays commits to the current HEAD. This change implements specific handling for commit verification failures: if verification fails, the operation no longer rolls back the transaction but instead persists the merge state, allowing users to resolve the failing tests and continue the operation using \--continue\. The implementation also supports amending previous commits, preserving the original author identity while using the current user for the committer identity, and provides options to control how empty commits are handled (erroring, dropping, or keeping).

_go/libraries/doltcore/cherry\pick · high confidence

The tabular output package now includes dedicated writers for displaying SQL row diffs and merge conflicts with improved formatting. The new FixedWidthDiffTableWriter renders row-level changes (added, removed, modified) using colored markers and supports inline text diffs for multiline values, while the FixedWidthConflictTableWriter formats conflict data by version (base, ours, theirs) with distinct color coding for each change type. These components rely on a refactored FixedWidthTableWriter that correctly handles multi-byte characters and multiline strings to ensure accurate column alignment in the terminal.

go/libraries/doltcore/table/untyped/tabular · high confidence

The remote storage layer now uses a new \ranges.Tree\ data structure to manage and prioritize remote data downloads. This tree efficiently coalesces adjacent byte ranges from the same URL into larger regions and maintains a heap to quickly select the next region to fetch based on configurable policies (largest, smallest, or random). This change replaces previous chunk-hedging logic with a more streamlined approach that leverages streaming, retries, and throughput monitoring to optimize download performance.

go/libraries/doltcore/remotestorage/internal/ranges · high confidence

The \go/libraries/utils/svcs\ package introduces a \Controller\ and \Service\ interface to manage the initialization, execution, and shutdown of background services. The controller registers services, runs their \Init\ methods in order, launches \Run\ methods concurrently, and handles \Stop\ in reverse order. A key behavioral change is that the controller's \Stop\ method is now safe to call concurrently and multiple times, and it never returns an error, simplifying shutdown logic for callers. The \Service\ interface passes a \RunState\ to \Stop\ so implementations can distinguish between cleanup after a failed initialization versus waiting for a running goroutine to finish.

go/libraries/utils/svcs · high confidence

The AWS remote now supports periodic refreshing of credentials when using explicit file-based profiles. A new utility in go/libraries/utils/awsrefreshcreds implements a credentials provider that wraps an existing provider and forces an expiration interval, ensuring that temporary credentials stored in files are automatically refreshed. This change accompanies the migration to the aws-sdk-go-v2 and ensures that long-running operations using file-based AWS credentials do not fail due to credential expiration.

go/libraries/utils/awsrefreshcreds · high confidence

Table name resolution now respects the PostgreSQL-style \search\path\ setting when enabled (primarily for Doltgres). The system parses the search path according to PostgreSQL rules (handling quoted identifiers, case folding, and whitespace) and resolves tables by iterating through the specified schemas. This change ensures that \dolt\\ prefixed system tables are correctly scoped to the \dolt\ namespace and remain accessible regardless of the user's search path, while also allowing Doltgres to override the effective user for \$user\ substitution in the search path.

go/libraries/doltcore/sqle/resolve · high confidence

The \go/store/prolly/message\ package has been refactored to introduce dedicated serializers for Prolly tree nodes, address maps, blobs, commit closures, and merge artifacts, all using a new \ItemAccess\ abstraction for efficient buffer access. A key behavioral change is that nodes now record the specific offsets of chunk addresses embedded within key and value tuples (such as out-of-band adaptive values) in new \KeyAddressOffsets\ and \ValueAddressOffsets\ fields. The \WalkAddresses\ function has been updated to traverse these recorded offsets, ensuring that all out-of-band data is correctly discovered and managed during garbage collection and push operations, while nodes without such addresses remain backward-compatible with older clients.

go/store/prolly/message · high confidence

The test driver utilities in go/libraries/doltcore/dtestutils/sql\_server\_driver have been restructured to improve reliability and cross-platform support. The code now separates platform-specific process handling into distinct files (cmd\_unix.go and cmd\_windows.go), ensuring that Windows tests correctly use process groups and CTRL\_BREAK\_EVENT for graceful server shutdowns, while Unix systems use SIGTERM. Additionally, the driver now supports dynamic port allocation and parallel test execution, and output logging has been refined to prepend server names and write to test logs for better readability in multi-server scenarios.

_go/libraries/doltcore/dtestutils/sql\_server\driver · high confidence

The \mvdata\ package has been rewritten to route table imports and exports through the SQL engine rather than using a legacy direct-write path. This change introduces new components such as \SqlEngineTableWriter\ and \SqlEngineTableReader\ to handle data movement, adds a \DataMoverPipeline\ for concurrent reading and writing, and implements a \ChannelRowSource\ to bridge channels with the SQL iterator interface. Users benefit from more consistent behavior, better error handling, and support for advanced import operations like partial column updates and foreign key constraints, all managed via the standard SQL execution layer.

go/libraries/doltcore/mvdata · high confidence

The reference handling logic in \go/libraries/doltcore/ref\ has been restructured to introduce stricter branch naming rules that align with Git semantics, rejecting names containing invalid characters, commit-id-like strings, or reserved tokens like 'HEAD'. The system now explicitly supports new reference types including Workspaces (\refs/workspaces/\), Stashes (\refs/stashes/\), Statistics (\refs/statistics/\), and Tuples (\refs/tuples/\), alongside the existing Branch, Tag, and Remote refs. Additionally, WorkingSet references have been moved out of the standard \refs/\ namespace to a dedicated \workingSets/\ prefix, and the refspec parsing logic has been updated to support wildcard mappings for remotes and tags.

go/libraries/doltcore/ref · high confidence

The row package has been restructured to introduce distinct row types for keyed and keyless tables, allowing the system to correctly handle tables without primary keys. This change includes a new row formatting function that displays column values in a readable, field-separated format, and updates to how NULL values are processed and printed to ensure they are handled consistently throughout the pipeline.

go/libraries/doltcore/row · high confidence

The session state management in the SQL engine has been refactored to improve branch handling and concurrency. A new \AutoIncrementTracker\ has been introduced to manage auto-increment values globally across branches, ensuring consistency during merges and resets. Session state is now organized around \DatabaseSessionState\ and \branchState\ structures, allowing for more granular tracking of branch heads, working sets, and caches. Additionally, branch control access checks have been refined to support merge-permission bypasses for conflict resolution, and transaction lifecycle hooks for DoltgreSQL have been integrated to better manage session events.

go/libraries/doltcore/sqle/dsess · high confidence

The Go bindings for the remotesapi service have been regenerated using the modern google.golang.org/protobuf library (replacing github.com/golang/protobuf) and updated protoc-gen-go tooling. This update introduces new API capabilities for clients and servers, including a \PushConcurrencyControl\ enum to manage branch push safety, a \Feature\ flag system to advertise support for the new \StreamChunkLocations\ RPC, and a \ClientCapability\ enum allowing clients to declare support for HTTP/2 downloads. Additionally, a new \CredentialsService\ with a \WhoAmI\ RPC has been added, enabling clients to retrieve their username, display name, and email address from the server.

go/gen/proto/dolt/services/remotesapi · high confidence

The Dolt remotestorage client now explicitly declares support for HTTP/2 downloads to the remote server. A new \capabilities.go\ file defines a \clientCapabilities\ list that includes \CLIENT\_CAPABILITY\_HTTP2\_DOWNLOAD\, and this capability set is automatically stamped onto all outbound remotesapi requests (such as \StreamChunkLocations\ and \GetDownloadLocations\) via the new \ChunkFetcher\. This ensures the server is aware of the client's ability to handle HTTP/2 range requests, enabling more efficient chunk downloads.

go/libraries/doltcore/remotestorage · high confidence

The project has officially rebranded from 'noms' to 'Dolt', updating the repository root README to reflect the new identity, logo, and documentation links. This change also introduces several new repository governance and configuration files: a CODEOWNERS file to define review responsibilities for integration tests, a SECURITY.md file establishing the policy for handling panics and reporting vulnerabilities, a LICENSE file adopting the Apache 2.0 license, and a .gitignore file updated to exclude modern IDE and AI-assisted development artifacts.

(repo-wide) · high confidence

The import benchmarker in go/performance/import\_benchmarker has been completely reworked to support configurable performance comparisons across different import targets (Dolt server, Dolt CLI, and external MySQL servers). Users can now define import tests via YAML files, specifying parameters such as file format (CSV or SQL), row counts, schema definitions, and shuffling options. The tool outputs performance metrics into a SQL table, allowing users to run comparative queries against import runtimes. This replaces the previous unit-test-based approach, which was not suitable for performance reporting, and introduces a more flexible, data-driven benchmarking experience.

_go/performance/import\benchmarker · high confidence

The \go/store/datas/pull\ package has been completely rewritten to replace the previous puller implementation. The new architecture introduces a \PullChunkTracker\ to efficiently manage and deduplicate chunk addresses, and a \PullChunkFetcher\ that supports streaming retrieval via a \GetManyCompressed\ interface, allowing remote chunk stores to optimize downloads. Data is now persisted to the destination using a \PullTableFileWriter\, which manages concurrent uploads of compressed table files with configurable target sizes and buffer limits. A new \Clone\ function enables full database cloning, including handling of journal files and concurrent table file downloads. The system also includes a \fetchStatsRecorder\ to capture detailed download statistics (bytes, retries, in-flight connections) for logging, and robust cleanup of temporary staging files on failure.

go/store/datas/pull · high confidence

The \dolt dump\ command now uses a new batched SQL writer that groups multiple row insertions into single statements (up to 10,000 per line), reducing output size and improving import performance. Additionally, a new SQL diff writer has been introduced, allowing \dolt diff\ to output changes as SQL data-diff statements (INSERT/UPDATE/DELETE) rather than the previous format. These changes are implemented in the \sqlexport\ package via \BatchSqlExportWriter\ and \SqlDiffWriter\.

go/libraries/doltcore/table/untyped/sqlexport · high confidence

The schema encoding layer in \go/libraries/doltcore/schema/encoding\ has been updated to support FlatBuffers serialization alongside the existing Noms format. This change introduces new \SerializeSchema\ and \DeserializeSchema\ methods for FlatBuffers, while \MarshalSchema\ and \UnmarshalSchema\ now act as wrappers that include failsafe validation of column constraints before serialization. Additionally, a cache for unmarshalled schemas has been added to improve performance when reading schemas from disk, and integration tests have been added to verify round-trip serialization for both formats.

go/libraries/doltcore/schema/encoding · high confidence

The \go/libraries/doltcore/schema\ package has been reorganized into distinct files for better maintainability. This change introduces \check\_coll.go\ to manage check constraints, \collation.go\ and \collation\_comparator.go\ to handle string collation logic, and \col\_coll.go\ to manage column collections. It also refactors \column.go\ to include new fields for generated and virtual columns, and updates \constraint.go\ to standardize constraint handling. These structural changes are accompanied by new tests in \check\_coll\_test.go\, \col\_coll\_test.go\, \collation\_comparator\_test.go\, and \constraint\_test.go\ to ensure the correctness of the new modular components.

go/libraries/doltcore/schema · high confidence

The table editor's configuration has been streamlined by removing unused fields and functions from the \EditorOpts\ structure. The \Options\ struct now contains only the \TargetStaging\ boolean, which controls whether edits target the staging root versus the working root. This change reduces the complexity of the editor's initialization and removes dead code related to previous implementation details.

go/libraries/doltcore/table/editor · high confidence

The default Noms data format version is now explicitly set to \_\DOLT\\_ (via the new \FormatDoltString\ constant in \go/store/constants/version.go\), replacing any previous implicit or obsolete defaults. This change ensures that new data stores and files created by the system will use the \_\DOLT\\_ format by default, aligning the constant definition with the corresponding format definitions in \go/store/types/format.go\.

go/store/constants · high confidence

The shim layer in go/store/prolly/shim now handles vector index nodes and keyless secondary schemas. When converting a serialized value to a prolly.Map or MapInterface, the code detects vector index nodes (identified by serial.VectorIndexNodeFileID) and constructs a NewProximityMap with appropriate distance and chunking metadata, rather than a standard map. Additionally, for keyless secondary indexes, the key descriptor is now modified using AddHashToSchema to ensure correct hashing behavior. This change enables the storage layer to correctly interpret and interact with vector-based indexes and keyless secondary structures.

go/store/prolly/shim · high confidence

The sqlserver package now uses a mutex-protected global variable to track the current running SQL server instance. This change introduces thread-safe accessors (GetRunningServer, SetRunningServer, UnsetRunningServer, and RunningInServerMode) to manage the server state, ensuring that concurrent checks or modifications to the server instance do not cause race conditions.

go/libraries/doltcore/sqlserver · high confidence

The \noms\ debugging and inspection commands (\show\, \manifest\, \root\, \walk\, \cat\, \ds\) have been updated to work with the new \Format\DOLT\ and \\\_DOLT\DEV\\_\ storage formats. This includes changes to how commit parents are stored and retrieved (supporting both \parents\_list\ and \parents\_closure\), improved handling of internal prolly tree nodes in \noms show\, and fixes to manifest parsing to accept both v4 and v5 formats. The \noms walk\ command now supports depth-first traversal of the new storage structure, and \noms cat\ has been updated to correctly decode values using the current database format.

go/store/cmd · high confidence

The copyright header check utility in go/utils/copyrightshdrs now accepts source files with copyright years extending to 2026, ensuring that recent code additions pass validation without requiring manual header updates.

go/utils/copyrightshdrs · high confidence

The gitauth utility now provides clearer feedback when authentication fails by wrapping errors in a NonInteractiveAuthError that includes specific hints for configuring credentials (e.g., using a credential helper for HTTPS or ssh-add for SSH). Additionally, Git process cancellation is more robust: on Unix, the entire process group is killed to prevent orphaned transport processes, and on Windows, taskkill is used to terminate descendants. A bounded wait delay prevents hanging when child processes outlive the main git command.

go/libraries/utils/gitauth · high confidence

The revert logic now aligns with Git by refusing to revert only when uncommitted changes affect tables that the revert actually touches, rather than blocking on all dirty tables. It also introduces support for multi-commit revert series, allowing users to interrupt, resume (via --continue), and abort operations while ensuring that an abort correctly undoes any commits already applied in the series.

go/libraries/doltcore/revert · high confidence

Add Elixir integration tests for MySQL/OTP and MyXQL connectors; Add Hibernate ORM smoke test for Dolt integration; Add MikroORM integration smoke test; Add RPM build infrastructure for Dolt; Add TypeORM smoke test for integration validation; Add test data for JWT authentication integration tests; Added ICU C++ library smoke test; Added Rust MySQL client integration test; Added SQL helper scripts for integration tests; Added baseline configuration for JSON old-format repository integration tests; Added benchmark suite for Prolly maps and BBolt comparisons; Added benchmark tests for dolt\_log performance; Added bidirectional compatibility integration tests for Dolt versions; Added bidirectional compatibility tests for remote column additions; Added compatibility tests for Dolt 2.0 adaptive encoding breaking changes; Added compatibility tests for descending indexes; Added corrupt database fixtures for fsck integration tests; Added integration tests for \dolt add --patch\ interactive behavior; Added integration tests for concurrent transaction isolation; Added integration tests for database revision, schema history, and procedure history tables; Added manifest clobber utility for testing; Added memory profiling benchmark for DoltDB loading; Added microbenchmark suite for sysbench-style OLTP queries; Added replication performance benchmarks; Added server performance benchmarks with CPU profiling; Added test data fixtures for import validation; Added test helpers for creating bare Git repositories; Added test utility package for file and reader mocking; Added test utility to compare strings ignoring hash values; Added tests for the store/spec package; BATS integration test suite stabilization and migration; Expanded compatibility test suite with new version lists and setup scripts; Expanded cross-version compatibility testing for data types and geometry; New Dolt SQL logic test harness; New Go-based integration test suite for dolt sql-server; New integration tests for MySQL dump loading; Updated GPG test key to remove expiration; Updated engine tests for go-mysql-server upgrade; Updated test fixtures for TLS certificates and configuration validation; Updated test fixtures for signed commits.

The \proto\ directory now uses Bazel for building and generating Go bindings from \.proto\ files. This introduces a new build configuration (MODULE.bazel, BUILD.bazel, .bazelrc) and a \//:update\ target that regenerates Go source files into the \../go/gen/\ directory. The migration updates the protobuf dependency to version 36.0.bcr.1 and sets the Go SDK version to 1.26.2.

proto · high confidence

This change updates the Go language version requirement and refreshes various Go module dependencies, including go-mysql-server, AWS SDK, gRPC, and others, to their latest compatible versions. These updates ensure the project uses current language features and library patches without introducing new user-facing features.

(dependencies) · high confidence

Added package documentation for libraries; Dolt version updated to 2.3.5.

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 58.

• Code Health 56
• Architecture 97
• Maturity 56
• Readiness 59
• Security 66
• Domain Modelling 66

• 300 commits — 239 feature/other, 61 fixes

• (repo) — 107 commits
• go/libraries — 71 commits
• go/store — 26 commits
• go/go.mod — 24 commits
• integration-tests/bats — 23 commits
• go/cmd — 14 commits
• .github/scripts — 13 commits
• .github/workflows — 9 commits
• .github/actions — 4 commits
• integration-tests/mysql-client-tests — 4 commits
• integration-tests/go-sql-server-driver — 3 commits
• go/performance — 1 commit
• integration-tests/compatibility — 1 commit

• fix: Enable the original overlapping-index commit regression
• fix: Fix CI scheduler permissions for PR metadata.
• fix: Fix copyright header.
• fix: Fix foreign keys with dropped columns
• fix: Fix keyless COUNT column projection
• fix: Fix legacy table name unmarshalling receiver
• fix: Fix range ordering issue in some situations
• fix: Fix reset replica filter lifecycle
• fix: Merge branch 'dolthub:main' into fix/existing-ref-error-ref-name
• fix: Merge generated column SQL diff fix.
• fix: Merge main into binary file regression branch
• fix: Merge pull request #11303 from vishnujayvel/fix/empty-query-warning-10864
• fix: Merge pull request #11666 from dolthub/fulghum/fix-auto-bump-close
• fix: Merge pull request #11693 from shiyudesu/fix/existing-ref-error-ref-name
• fix: Merge pull request #11757 from dolthub/aaron/byte-sync-boundary-condition-fix
• fix: Merge pull request #11796 from dolthub/aaron/journal-writer-close-race-fix
• fix: Merge pull request #11802 from dolthub/db/fix-tablename
• fix: Merge pull request #11808 from dolthub/zachmu/issue-4501-regression-test
• fix: Merge pull request #11811 from dolthub/zachmu/issue-5414-regression-test
• fix: Merge pull request #11814 from dolthub/zachmu/issue-6124-regression-test
• …and 280 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

dolthub/dolt was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.