neondatabase/neon

This system is a distributed cloud-native PostgreSQL platform that decouples compute and storage to enable elastic scaling and high availability. It manages the lifecycle of PostgreSQL compute nodes, which interact with a sharded, tiered storage layer composed of pageservers and safekeepers for data persistence and replication. The architecture includes a control plane for orchestrating these components, a proxy for routing and authenticating client connections, and a suite of specialized libraries for handling WAL processing, metrics, and remote storage integration.

55 changes.

This period established the foundational structure of the Neon project, initializing the repository with build tooling, dependency management, and core Rust components. It introduced the primary services—pageserver, safekeeper, and proxy—along with their respective HTTP APIs, authentication systems, and modular control planes. The work also laid the groundwork for testing and observability by implementing comprehensive test suites, benchmarking infrastructure, and shared utility libraries for storage and protocol handling.

96 changes.

This period focused on establishing the storage controller as the central coordination layer for tenant shard placement and safekeeper management, while modernizing the pageserver with Rust-based WAL processing, tiered compaction, and direct I/O support. Significant architectural shifts included migrating the proxy to a modular, serverless-ready design with dynamic rate limiting and vendored protocol libraries, alongside introducing comprehensive benchmarking and simulation tools for performance validation.

30 changes.

This period focused on migrating core inter-service communication from libpq to a high-performance gRPC interface for the pageserver, alongside a comprehensive rewrite of the proxy's TLS handling using rustls. Significant effort was also directed toward expanding the compute\_tools HTTP management API and establishing new infrastructure for Local File Cache and safekeeper scheduling policies.

The proxy now supports an Additive-Increase Multiplicative-Decrease (AIMD) algorithm for dynamic rate limiting. This new algorithm adjusts the concurrency limit based on utilization and error outcomes: it increases the limit additively when successes are observed at high utilization, and decreases it multiplicatively when overload errors occur. This provides a congestion-avoidance mechanism for managing proxy load.

_proxy/src/rate\_limiter/limit\algorithm · high confidence

The \pgxn/neon\_rmgr\ directory now contains the source files for the \neon\_rmgr\ extension, introducing a custom WAL resource manager for PostgreSQL 16 and later. This extension registers a new resource manager (\RM\_NEON\_ID\) that handles Neon-specific WAL records for heap operations (insert, delete, update, lock, and multi-insert). It provides the necessary infrastructure for redoing these operations during crash recovery, masking page data to ensure consistent replay, and decoding WAL records for logical replication output plugins.

_pgxn/neon\rmgr · high confidence

Added benchmarking infrastructure to the safekeeper crate, including a README with usage instructions and a \receive\_wal.rs\ benchmark file. These benchmarks measure Safekeeper performance for WAL ingestion, specifically testing \process\_msg\ latency and \WalAcceptor\ throughput under varying conditions (fsync, commit, message size, batch count). The benchmarks use Criterion for statistical analysis and pprof for flamegraph profiling, and are configured to use jemalloc to mirror production settings.

safekeeper/benches · high confidence

Added a new benchmarking setup for the \utils\ library, including a \README.md\ with instructions for running benchmarks via \cargo bench\ and a \benchmarks.rs\ file that defines Criterion benchmarks. These benchmarks measure the performance of \id::TenantTimelineId\ stringification and the \log\_slow\ logging mechanism (with and without the feature enabled), supporting profiling via pprof-rs.

libs/utils/benches · high confidence

A new \Duplex\ channel utility has been added to the \libs/utils\ library, providing a convenient wrapper around Tokio's \mpsc\ channels to facilitate bi-directional communication between two endpoints. This allows components to send and receive data simultaneously over a single logical connection, simplifying the setup of request-response or streaming patterns where both sides need to transmit data concurrently.

libs/utils/src/sync/duplex · high confidence

The storage controller now persists scrubber scan results in a new \metadata\_health\ database table. This table records the health status and last scrubbed timestamp for each tenant shard, enabling the system to track the state of metadata health checks over time.

_storage\_controller/migrations/2024-07-23-191537\_create\_metadata\health · high confidence

The new neon\_test\_utils extension (version 1.3) provides a suite of helper functions for Neon testing and debugging. It includes utilities to consume XIDs, OIDs, CPU, and memory (with explicit release), clear the buffer cache, retrieve raw pages at a specific LSN, flush WAL, and trigger panics or segfaults for fault-injection testing.

_pgxn/neon\_test\utils · high confidence

Adds a new benchmark suite for the pageserver, including \bench\_layer\_map\ to measure layer map search performance against real and synthetic data, \bench\_walredo\ to quantify WAL redo throughput under concurrent load, \bench\_ingest\ to test data ingestion performance with configurable key layouts and I/O modes, and \bench\_metrics\ to demonstrate and measure multicore scalability bottlenecks in the metrics library.

pageserver/benches · high confidence

The \neon-shmem\ library now provides a resizable hash map (\HashMapInit\/\HashMapAccess\) and a shared-memory-aware read-write lock (\PthreadRwLock\). The hash map supports dynamic growth and shrinkage within a contiguous shared memory area (backed by \memfd\ via \ShmemHandle\), allowing multiple processes to share and modify the same data structure safely. It includes standard map operations, an \Entry\ API for efficient insert/lookup, and concurrency control via a single shared-memory RwLock guarding the entire map.

libs/neon-shmem · high confidence

A new script and accompanying Ansible playbook have been added to safely remove tenant directories from safekeepers for projects that have been deleted in the console. The script verifies deletion status via the console API, moves tenant data to a trash directory for safety, and then removes it from the safekeeper, supporting both dry-run testing and automated execution via Ansible.

_scripts/sk\_cleanup\tenants · high confidence

New Python utilities have been added to the \test\_runner/performance/benchbase\_tpc\_c\_helpers\ directory to support TPC-C performance testing. \generate\_workload\_size.py\ creates BenchBase XML configuration files and execution scripts for warmup, ramp-up, and benchmark phases. \generate\_diagrams.py\ produces SVG charts visualizing throughput (TPS) and latency percentiles from CSV results. \upload\_results\_to\_perf\_test\_results.py\ extracts metrics from BenchBase JSON/CSV outputs and inserts them into the \perf\_test\_results\ and \benchbase\_results\_details\ PostgreSQL tables for tracking.

_test\_runner/performance/benchbase\_tpc\_c\helpers · high confidence

A new SVG visualization module has been added to the compaction simulator to help users inspect layer history. This tool generates diagrams that map key ranges and LSN (Log Sequence Number) ranges to coordinates, rendering delta and image layers as rectangles. The visualization logic ensures image layers are drawn on top of delta layers when they overlap, providing a clear view of the compaction state.

pageserver/compaction/src/simulator · high confidence

New SQL scripts and a Python loader have been added to the test runner to benchmark pgvector index construction and query performance. The suite now includes specific tests for HNSW and IVFFlat indexes across multiple distance metrics (cosine, inner product, L2, Hamming, Jaccard) and introduces support for half-precision vector (halfvec) indexing and querying via pgbench scripts. These changes allow users to validate performance characteristics of these indexing strategies against a 1M-row dataset.

_test\runner/performance/pgvector · high confidence

A new stub library named subzero\_core has been added to the proxy libs directory. This includes a basic Rust library structure with a .gitignore file and a lib.rs file containing a comment indicating it is a placeholder for the subzero-core crate.

_libs/proxy/subzero\core · high confidence

Generated type stubs (\.pyi\ files) for the \h2\ HTTP/2 library have been added to \test\_runner/stubs/h2\. This provides static type checking support for the library's core components, including connection management, stream handling, configuration, events, and error codes, improving type safety for code that interacts with HTTP/2.

_test\runner/stubs · high confidence

The new compute.sh script in the docker-compose environment now automatically handles the setup of the compute node by waiting for the pageserver, and dynamically creating or retrieving tenant and timeline IDs via the pageserver API if they are not provided. It also configures the PostgreSQL instance by injecting the appropriate ULID extension (pgx\_ulid for PG 17+, ulid otherwise) into the shared preload libraries and updates the configuration file with the resolved tenant and timeline IDs before launching compute\_ctl.

_docker-compose/compute\wrapper/shell · high confidence

The sql\_exporter configuration now exposes a broad set of new metrics to improve observability of the compute node. Local File Cache (LFC) usage and performance are tracked via \lfc\_used\, \lfc\_hits\, \lfc\_misses\, \lfc\_writes\, and latency histograms (\file\_cache\_read\_wait\_seconds\, \file\_cache\_write\_wait\_seconds\). Replication health is enhanced with \replication\_delay\_bytes\, \replication\_delay\_seconds\, \retained\_wal\, and \wal\_is\_lost\ metrics. PostgreSQL internal state is now visible through \compute\_pg\_oldest\_frozen\_xid\_age\, \compute\_pg\_oldest\_mxid\_age\, \compute\_max\_connections\, \compute\_subscriptions\_count\, and \compute\_logical\_snapshots\_bytes\. Additionally, low-level Neon performance counters for getpage requests, prefetching, and pageserver interactions are exported, and checkpoint metrics are updated to support PostgreSQL 17.

_compute/etc/sql\exporter · high confidence

The repository is initialized with the foundational structure for the Neon project, including a Dockerfile for building storage service images, a Makefile for compiling PostgreSQL (versions 14–17) and Rust components, and a .dockerignore to optimize build contexts. It introduces a pre-commit hook (pre-commit.py) for Rust and Python formatting, configuration for cargo-deny to enforce license and dependency policies, and a CODEOWNERS file to assign review responsibilities across compute, proxy, and storage teams.

(repo-wide) · high confidence

The remote storage library now supports Google Cloud Storage alongside the existing AWS S3, Azure Blob, and local filesystem backends. This change adds a new \GCS\ variant to the \RemoteStorageKind\ configuration enum and implements the \GCSBucket\ client, allowing users to configure and use GCS buckets for timeline and WAL offloading. The implementation includes GCS-specific configuration options (bucket name, prefix, concurrency limits) and integrates with the existing remote storage metrics and error handling systems.

_libs/remote\storage/src · high confidence

Added a new lightweight PostHog client library that enables local evaluation of feature flags, removing the need for synchronous remote API calls during flag checks. This library includes a background loop that periodically fetches flag definitions from PostHog and updates an in-memory store, allowing the system to evaluate boolean flags and multivariate rollouts locally using a consistent hashing algorithm. It also supports reporting telemetry events, such as fake tenant properties, to PostHog for observability purposes.

_libs/posthog\_client\lite · high confidence

The proxy now includes a new SASL (Simple Authentication and Security Layer) module that handles the authentication protocol exchange. This change adds support for channel binding flags (parsing and encoding GS2 headers) and defines the message structures for the initial SASL handshake. It introduces a generic \Mechanism\ trait and an \authenticate\ stream handler that manages the multi-step challenge-response loop, allowing specific mechanisms like SCRAM to be plugged in. This provides the foundational infrastructure for secure, extensible client authentication within the proxy.

proxy/src/sasl · high confidence

A new \pagebench\ CLI tool is added to the pageserver component to provide component-level performance testing. It supports multiple benchmark subcommands including basebackup, getpage\_latest\_lsn, trigger\_initial\_size\_calculation, ondemand\_download\_churn, aux\_files, and idle\_streams. The tool integrates tracing for logging, replaces the default panic hook with a tracing-aware one, and includes optional CPU profiling support that generates SVG flamegraphs upon completion.

pageserver/pagebench/src · high confidence

A new \compute\_api\ library has been added to define the shared data structures and API contracts for the compute control plane. This includes the \ComputeSpec\ struct, which details how to start PostgreSQL and connect to storage nodes (including tenant/timeline IDs, pageserver connection info, and feature flags), as well as request/response types for \compute\_ctl\'s HTTP endpoints (such as configuration, extension installation, and role grants). It also defines JWT claim structures for authorization and enumerations for privileges and compute status states.

_libs/compute\api/src · high confidence

The \compute\_tools\ crate has been added to \compute\_tools/src\, consolidating the core logic for managing compute nodes. This includes \catalog.rs\ for retrieving database and role information and dumping schemas, \checker.rs\ for writability checks, \compute.rs\ for the main \ComputeNode\ lifecycle and configuration, \compute\_prewarm.rs\ for Local File Cache (LFC) prewarming, and \compute\_promote.rs\ for handling replica promotion. The crate also introduces configuration management via \config.rs\ and \configurator.rs\, along with templates for rsyslog-based log export. This change represents the structural foundation for the compute control plane's interaction with the Postgres instance.

_compute\tools/src · high confidence

The compute directory now includes a \manifest.yaml\ file that defines default PostgreSQL settings (such as connection checks, WAL behavior, and logging verbosity) and a JSON schema (\manifest.schema.json\) to validate these configurations. Additionally, the build system has been updated to use Jsonnet templates to generate configuration files for \sql\_exporter\ and \neon\_collector\, replacing previous static or ad-hoc configuration methods. This change centralizes default compute behavior and ensures configuration consistency across different PostgreSQL versions and deployment targets.

compute · high confidence

A new JSON serialization library has been added to \libs/proxy/json\ to provide more flexibility and performance than standard \serde\_json\. It supports dynamic construction of JSON values without requiring serde-aware models, allowing users to build JSON incrementally to reduce memory overhead and CPU lag spikes. The library includes macros for easy string and vector generation, handles async streams, and implements custom encoding for primitives, strings, and collections.

libs/proxy/json · high confidence

The serverless proxy now uses a new \PoolingBackend\ and dedicated connection pools (\LocalConnPool\, \HttpConnPool\) to manage SQL-over-HTTP and local proxy connections. This change adds specific modules for handling connection lifecycle, cancellation (\CancelSet\), and error classification (\ConnInfoError\, \ReadPayloadError\), enabling more granular control over connection reuse, idle timeouts, and resource cleanup for serverless workloads.

proxy/src/serverless · high confidence

A new \endpoint\_storage\ service has been added to provide an API for uploading, downloading, and deleting files used by compute and control plane for LFC (Local File Cache) prewarm data. The service exposes REST endpoints for GET, PUT, and DELETE operations scoped by tenant, timeline, and endpoint IDs, along with prefix deletion capabilities. It enforces JWT-based authentication using EdDSA keys and supports configuration via file path or inline JSON, with a default listening port of 51243. An OpenAPI specification is included to document the API contract.

_endpoint\storage · high confidence

A new \fast\_import\ binary is added to \compute\_tools\ to support high-performance database imports into Neon timelines. This tool provides two modes: \Pgdata\, which runs a local Postgres instance to restore a dump and uploads the resulting PGDATA to S3, and \DumpRestore\, which performs a direct \pg\_dump\ to \pg\_restore\ between source and destination databases. The implementation includes support for configurable CPU and memory resources, encrypted connection string handling via KMS, and parallel S3 uploads for efficient data transfer.

_compute\tools/src/bin · high confidence

The pageserver now exposes a new gRPC interface for compute-to-pageserver communication, replacing the previous libpq-based path for page reads. This change introduces a Protobuf schema (\page\_service.proto\) and a Rust client library that supports bidirectional streaming for \GetPages\ to improve throughput, as well as unary RPCs for base backups, database/relation sizes, SLRU segments, and LSN leases. The API includes built-in authentication via metadata interceptors, optional payload compression for base backups, and a \GetPageSplitter\ to handle requests spanning multiple shards.

_pageserver/page\api · high confidence

The pageserver now includes new \blob\_io.rs\ and \block\_io.rs\ modules that define the internal format for reading and writing variable-sized data blobs (including zstd compression support) and provide a \BlockCursor\ abstraction for accessing 8 KB pages from underlying storage. These changes establish the foundational I/O primitives used by the layer map and read paths to handle compressed and uncompressed data efficiently.

pageserver/src/tenant · high confidence

The Neon extension is now distributed as a standard PostgreSQL extension (pgxn/neon) that can be installed into any database and supports schema changes via relocation. It bundles core storage components including the local file cache (LFC), pagestore communication, WAL proposer, and logical replication monitoring. Additionally, a dedicated background worker process now exposes a Prometheus metrics endpoint via a Unix domain socket, providing visibility into LFC performance and other internal statistics.

pgxn/neon · high confidence

The proxy now supports a new authentication backend named \cplane\_proxy\_v1\. This change introduces a new implementation of the control plane client (\NeonControlPlaneClient\) in \proxy/src/control\_plane/client/cplane\_proxy\_v1.rs\ and registers it as a variant in the \ControlPlaneClient\ enum within \mod.rs\. The new backend handles fetching endpoint access control, role access control, and JWKS from the control plane, utilizing the \clashmap\ library for rate-limiting locks and supporting features like IP allowlists and VPC endpoint checks. A corresponding mock implementation (\MockControlPlane\) is also added for testing purposes.

_proxy/src/control\plane/client · high confidence

This change adds the \pageserver/pagebench/src/util\ module, providing core utilities for the new component-level benchmarking tool. It includes \targets.rs\ for discovering and limiting tenant/timeline targets via the management API, \request\_stats.rs\ for tracking request latency using a fixed-bounds histogram (reporting mean and p95/p99/p99.9/p99.99 percentiles), and \tokio\_thread\_local\_stats.rs\ for managing thread-local statistics across a Tokio runtime.

pageserver/pagebench/src/util · high confidence

Adds a new \test\_runner/fixtures/pageserver\ package to centralize test infrastructure for the pageserver. This includes an \allowed\_errors.py\ module that provides a configurable allow-list for expected pageserver and storage controller log errors (e.g., shutdown races, connection resets, and S3 retries), along with a CLI tool to scan logs against these rules. The package also introduces \common\_types.py\ for parsing layer file names and index metadata, \http.py\ with typed dataclasses for API responses (tenants, timelines, layers), and utility modules (\remote\_storage.py\, \many\_tenants.py\, \utils.py\) to support tenant duplication, remote storage operations, and state-waiting helpers for tests.

_test\runner/fixtures/pageserver · high confidence

The pageserver now supports importing PostgreSQL data directories (PGDATA) from remote storage (S3 or local filesystem) into an empty root timeline. This change introduces a new import flow that is sharding-aware, meaning it produces image layers containing only the data relevant to the specific shard. The import process is resumable: it tracks progress via a storage controller, allowing imports to survive tenant shutdowns and restarts without losing state. The implementation includes a planner that generates deterministic import plans, a remote storage wrapper for listing and downloading files, and format definitions for tracking import status (InProgress/Done) and idempotency keys.

_pageserver/src/tenant/timeline/import\pgdata · high confidence

Added the core infrastructure for secondary pageserver tenants to maintain local layer copies via periodic heatmap exchanges. This includes a new \heatmap\_uploader\ task that periodically serializes tenant layer metadata (including generation numbers and access times) and uploads it to remote storage, and a \downloader\ task that fetches these heatmaps and schedules layer downloads based on the heatmap data. A generic \scheduler\ component manages the concurrency and periodicity of these background jobs, supporting jitter and warmup periods to prevent thundering herd issues.

pageserver/src/tenant/secondary · high confidence

The storage controller is now available as a standalone service, providing the central coordination layer for pageservers and safekeepers. This release adds the core infrastructure for managing tenant shard placement, including background operations for draining and filling nodes, a heartbeat mechanism to monitor node availability, and a leadership protocol for high-availability failover. It also introduces HTTP endpoints for tenant and timeline management, compute notification hooks, and comprehensive metrics and authentication (JWT) support.

_storage\controller/src · high confidence

The storage scrubber tool is now available to audit and clean remote storage. It validates pageserver and safekeeper metadata consistency (checking index parts, layer maps, and tenant manifests), identifies and purges garbage objects (including those left by known deletion bugs or marked deleted in the console), performs physical garbage collection of old indices and ancestor layers, and scans for large objects. The tool supports concurrent operations, configurable concurrency limits, and can post health status to the storage controller.

_storage\scrubber/src · high confidence

This change introduces the \storage\_broker\ component, a new gRPC service built with Tonic and Tokio that replaces etcd for internal storage node communication. It provides a pub-sub model for safekeeper updates, allowing subscribers to listen to specific timelines or all messages via a filter. The service includes a binary implementation with HTTP/2 keepalive support, TLS configuration, and metrics for monitoring publisher/subscriber counts and message throughput. A benchmarking tool is also added to measure requests per second under load.

_storage\broker · high confidence

Adds a new command-line interface (\storcon\_cli\) to manage the storage controller, enabling administrators to register and configure pageservers, manage tenant lifecycle (create, delete, split, migrate shards), adjust tenant placement and scheduling policies, and monitor node and tenant states.

_control\_plane/storcon\cli · high confidence

The \libs/postgres\_connection\ library now provides a new \PgConnectionConfig\ struct to manage PostgreSQL connection settings, replacing previous URL-based or string-based configuration approaches. This change introduces a type-safe interface for specifying host, port, password, and connection options, with built-in security measures to prevent accidental password leakage in logs or debug output. The configuration supports building \tokio\_postgres\ client configurations and includes utilities for parsing host-port strings, enabling more robust and maintainable database connection management within the system.

_libs/postgres\connection · high confidence

Added a new library for calculating synthetic storage size based on a tree of timeline segments, snapshots, and WAL retention. The \calculate\ method determines the most cost-effective way to retain required history (using snapshots or WAL) and reports a minimum size of 1 to ensure offloaded tenants appear in monitoring. Additionally, an SVG visualization module was added to graphically represent the timeline branches, snapshot points, and retained WAL segments.

_libs/tenant\_size\model/src · high confidence

Adds the \vm-monitor\ library and binary, a core component of the autoscaling system that manages Postgres file cache sizing and cgroup memory limits to handle upscaling and downscaling decisions. The monitor exposes an Axum HTTP server with a WebSocket endpoint (\/monitor\) for communication with autoscaler agents, negotiating protocol versions and dispatching messages to coordinate resource adjustments.

_libs/vm\monitor · high confidence

The pageserver now includes a new \aligned\_buffer\ module providing \AlignedBuffer\, \AlignedBufferMut\, and \RawAlignedBuffer\ types that enforce memory alignment requirements. These types support both compile-time and runtime alignment specifications and integrate with \tokio\_epoll\_uring\ to enable direct I/O operations, laying the groundwork for optimized delta and image layer reads and writes.

_pageserver/src/virtual\_file/owned\_buffers\_io/aligned\buffer · high confidence

The \libs/postgres\_backend\ crate has been introduced to provide a server-side asynchronous Postgres connection handler. This new component manages the Postgres protocol state machine, including TLS encryption, authentication (including JWT), and query processing via a generic \Handler\ trait. It replaces the previous synchronous implementation, allowing for non-blocking I/O operations and improved resource management during connection lifecycle events.

_libs/postgres\backend/src · high confidence

A new shared library for collecting consumption metrics has been added, introducing structured event types for both absolute timestamps and incremental time ranges. To ensure reliable data transmission, the library implements an idempotency key mechanism that combines the current UTC time, a node identifier, and a random nonce, allowing downstream systems to detect and handle upload retries. Additionally, the library defines a chunking constant of 1000 metrics per batch to prevent exceeding maximum request size limits during serialization.

_libs/consumption\metrics · high confidence

This change introduces new data models in the pageserver API to support advanced scheduling and timeline management. It adds \AncestorDetached\ to track timelines reparented during ancestor detachment, \Partitioning\ to expose separated dense and sparse key spaces with their associated LSN, and \PageserverUtilization\ to provide disk usage, shard counts, and a calculated utilization score for scheduling decisions. These models enable the storage controller to make more informed placement decisions and allow clients to inspect the detailed state of timeline partitions and pageserver load.

_libs/pageserver\api/src/models · high confidence

The proxy now exposes a dedicated HTTP health server (proxy/src/http/health\_server.rs) that provides a /v1/status endpoint for health checks, a /metrics endpoint for Prometheus-compatible metrics (replacing the previous Prometheus handler with the measured library), and new /profile/cpu and /profile/heap endpoints to enable CPU and heap profiling. This server is wired into the proxy's HTTP module (proxy/src/http/mod.rs), which also consolidates HTTP client creation with OpenTelemetry tracing and retry policies.

proxy/src/http · high confidence

The \compute\_tools\ module now exposes a new HTTP server (\compute\_ctl\) that manages compute lifecycle and configuration. This server distinguishes between internal (local) and external (control plane) interfaces, with the external side secured via JWT authorization. Key capabilities include querying status and metrics, managing LFC prewarming and offloading, promoting replicas, configuring the compute, and installing extensions. The API is documented via an OpenAPI specification and includes request ID propagation and OTEL tracing for observability.

_compute\tools/src/http · high confidence

The \libs/tracing-utils\ crate now provides a unified entry point for initializing OpenTelemetry tracing infrastructure, including explicit configuration via \init\_tracing\ and \init\_performance\_tracing\ functions that support environment-variable-based export settings. It introduces a new \http\ module that wraps Hyper HTTP requests in OpenTelemetry spans, automatically extracting and propagating trace context from incoming headers and recording HTTP method, URI path, and status code. Additionally, a \perf\_span\ module offers a custom \PerfSpan\ type and instrumentation trait to route performance-critical spans to a dedicated subscriber, minimizing overhead for sampled tracing.

libs/tracing-utils · high confidence

A new Rust client library has been added to the \pageserver/client\ crate, providing structured interfaces for interacting with the Pageserver. This includes an HTTP-based management API client (\mgmt\_api\) for operations such as listing tenants, retrieving tenant and timeline details, and accessing the keyspace, as well as a PostgreSQL-based page service client (\page\_service\) supporting the V3 protocol for page stream requests and basebackup downloads. The library also exposes a \BlockUnblock\ enum to distinguish between GC blocking and unblocking operations.

pageserver/client · high confidence

Added a suite of SQL scripts to the compute tools to handle specific initialization and maintenance tasks. These include creating a \health\_check\ table for availability monitoring, managing \databricks\_reader\ role timeouts, and establishing a configurable privileged role. The changes also introduce logic to revoke privileges from roles before dropping them to prevent dangling permissions, ensure the \public\ schema is owned by the database owner, disable and drop logical replication subscriptions, and unset the template flag on databases prior to deletion.

_compute\tools/src/sql · high confidence

A new \wal\_craft\ library has been added to \libs/postgres\_ffi\ to facilitate testing of WAL (Write-Ahead Log) utilities. This library provides a \Crafter\ trait and concrete implementations (such as \Simple\ and \WalRecordCrossingSegmentFollowedBySmallOne\) to generate specific WAL patterns. It includes infrastructure to spin up temporary PostgreSQL instances (\PostgresServer\), run \initdb\, and execute \pg\_waldump\ to verify that the \find\_end\_of\_wal\ utility correctly identifies the end of WAL against crafted data. The module also contains specific unit tests for checkpoint logic, such as \test\_update\_next\_xid\.

_libs/postgres\_ffi/wal\craft/src · high confidence

A new command-line tool, \wal\_craft\, has been added to the \postgres\_ffi\ library to help generate and test specific Write-Ahead Log (WAL) patterns. Users can now craft WAL entries with specific properties (such as simple records, XLOG\_SWITCH records, or records crossing segment boundaries) either in a newly initialized PostgreSQL data directory or against an existing database instance. The tool supports multiple PostgreSQL versions and provides subcommands to print required configuration settings, initialize a new server with \initdb\, or connect to an existing server via a connection string.

_libs/postgres\_ffi/wal\craft/src/bin · high confidence

The \libs/wal\_decoder\ crate has been introduced to centralize the decoding and interpretation of PostgreSQL WAL records. This library extracts raw WAL bytes into structured \InterpretedWalRecord\ objects, filtering data by shard identity so that each shard only receives the keys it owns (with shard zero observing metadata for relation size tracking). The decoded records are serialized into \SerializedValueBatch\ structures and can be transmitted using either Bincode or Protobuf formats (with optional Zstd compression), enabling efficient, sharded ingestion of WAL data by the pageserver.

_libs/wal\decoder · high confidence

The storage controller now includes a ChaosInjector that periodically injects faults such as forced controller exits and tenant migrations to improve resilience testing. A new FeatureFlagService periodically fetches feature flag specifications from PostHog and propagates them to all pageservers. Additionally, the controller introduces a SafekeeperReconciler to manage safekeeper-specific operations in parallel, along with dedicated services for safekeeper timeline creation, migration, and membership management, enabling more robust handling of safekeeper topology changes.

_storage\controller/src/service · high confidence

A new Dockerfile for the compute wrapper test environment has been added, based on the compute-node-v14 image. This image installs curl, jq, and netcat, configures apt/wget/curl retries, and prepares directories for pg\_hint\_plan, file\_fdw, and PostGIS extensions to support extension testing. Additionally, a new Ed25519 key pair (private-key.pem and public-key.pem) has been introduced in the compute\_wrapper directory, likely for signing or verification purposes within the test infrastructure.

_docker-compose/compute\wrapper · high confidence

The compute\_tools service now exposes a comprehensive set of HTTP routes to manage the compute instance's lifecycle and configuration. Users can now trigger compute startup and reconfiguration via the /configure endpoint, which accepts a JSON spec and waits for the compute to reach a Running state. The /terminate endpoint allows for graceful shutdown, returning the final LSN and handling edge cases like empty computes. Replica promotion is now supported via the /promote endpoint, accepting a spec and WAL flush LSN. Additional operational endpoints include /status for state retrieval, /check\_writability to verify readiness, /get\_schema\_dump and /get\_dbs\_and\_roles for catalog inspection, and /install\_extension for adding extensions. The API also supports LFC (Local File Cache) management with /prewarm, /offload, and /cancel\_prewarm, as well as /refresh\_configuration to signal the compute to pull new specs. Metrics are exposed via /metrics (Prometheus format) and /metrics\_json, with a dedicated /autoscaling\_metrics endpoint forwarding data from the Postgres neon extension. A liveness probe is available at /hadron\_liveness\_probe, and failpoints can be configured for testing via /configure\_failpoints.

_compute\tools/src/http/routes · high confidence

The \libs/utils/src/sync\ module now includes three new synchronization primitives. The \Gate\ provides a mechanism for safe shutdown by allowing resources to acquire guards that prevent the gate from closing until all holders are released, with observability logging for delays. \HeavierOnceCell\ offers a custom once-cell implementation using \tokio::sync::Semaphore\ that supports \take\_and\_deinit\ without holding an outer mutex guard during initialization, ensuring panic and cancellation safety. \SpscFold\ introduces a single-producer single-consumer channel that allows the sender to fold values before transmission, handling backpressure and state transitions between sender and receiver.

libs/utils/src/sync · high confidence

The control\_plane directory now includes a README.md clarifying that neon\_local is a development and testing tool rather than a production system, along with example commands for initializing a local environment with Postgres 16. Additionally, new configuration files (safekeepers.conf and simple.conf) define default settings for pageservers and safekeepers, including network addresses and authentication types, to support local testing scenarios.

_control\plane · high confidence

Added a new \desim\ library that provides a deterministic, virtual-time simulation framework for distributed systems. It allows nodes to run as separate threads within a simulated environment, enabling fast-forwarding of time to skip idle intervals and injecting network failures without waiting for real-time timeouts. This tool is designed to test consensus implementations (such as walproposer and safekeepers) in realistic failure scenarios more efficiently than traditional integration tests.

libs/desim · high confidence

Added a new suite of performance tests under \test\_runner/performance/large\_synthetic\_oltp\ and \test\_runner/performance/many\_relations\ to evaluate Neon under complex, high-cardinality workloads. The large synthetic OLTP tests simulate realistic application schemas (including webhooks, workflows, and transaction logs) with heavy insert, update, and Zipfian-distributed read patterns to stress the storage engine. The many-relations tests generate hundreds of partitioned tables to benchmark the system's ability to handle a high number of database objects, similar to control-plane operations. These tests are supported by a new \out\_dir\_to\_csv.py\ utility for converting JSON benchmark results into CSV format for analysis.

_test\runner/performance · high confidence

A new PostgreSQL extension named neon\_utils has been added, providing a utility function num\_cpus() that returns the number of online processors on the host system. This allows users to programmatically detect CPU count within SQL queries, with the implementation handling both Linux (via sysconf) and Windows (via GetSystemInfo) environments.

_pgxn/neon\utils · high confidence

The neon\_walredo extension is introduced under pgxn, providing a standalone helper process that performs WAL redo operations. It includes an in-memory storage manager (inmem\_smgr) to handle page buffers without persistent storage, and integrates seccomp BPF filtering to restrict system calls for security. The process communicates via stdin/stdout using a custom protocol (BeginRedoForBlock, PushPage, ApplyRecord, GetPage, Ping) and supports both x86\_64 and aarch64 architectures.

_pgxn/neon\walredo · high confidence

The pagebench tool now includes new benchmarking commands for auxiliary file ingestion and listing, base backup retrieval (supporting both libpq and gRPC protocols with optional compression), and opening large numbers of idle gRPC GetPage streams. These additions expand the performance testing capabilities for pageserver operations beyond the existing getpage benchmarks.

pageserver/pagebench/src/cmd · high confidence

Added a new directory for pageserver performance tests, including a README with instructions for running benchmarks on EC2 instances with Instance Store, a new test file \test\_page\_service\_batching.py\ that measures batching factor and throughput under various pipelining configurations (serial vs. pipelined, scattered LSNs), and utility functions in \util.py\ to set up pageserver environments with multiple tenants and ensure they are ready for benchmarking by waiting for all tenants to become active and reconciling until idle.

_test\runner/performance/pageserver · high confidence

The metrics library now includes a HyperLogLog implementation for approximate distinct-count tracking, a LaunchTimestamp gauge to identify process restarts, and additional Linux process metrics (virtual memory lock and high-resolution CPU seconds). It also introduces I/O wrapper types (CountedReader, CountedWriter) to easily instrument byte-level read/write activity, and adds an InfoMetric type for exposing static label-based information metrics.

libs/metrics · high confidence

Added a new \scripts/sk\_collect\_dumps\ tool that automates the collection of state dumps from safekeeper nodes and their ingestion into a PostgreSQL database. The script uses Ansible playbooks to fetch JSON debug data via the \/v1/debug\_dump\ API, supporting both AWS SSM and Teleport (\tsh\) for remote access. It then processes these JSON files using \psql\ to insert the data into a temporary table and finally creates a typed output table with specific columns for safekeeper metrics (such as LSNs, memory state, and timeline IDs).

_scripts/sk\_collect\dumps · high confidence

Added \restore\_from\_wal.sh\ and \restore\_from\_wal\_initdb.sh\ to handle database restoration from Write-Ahead Logs. The primary script automates the \initdb\ process using the \cloud\_admin\ role and \C.UTF-8\ locale, with logic to select the correct \--locale-provider\ flag based on the Postgres version (libc for 15/16, builtin for 17). It configures the data directory, starts the server to apply WAL segments, and stops it immediately, supporting Postgres versions 14 through 17.

libs/utils/scripts · high confidence

Added a suite of new scripts to the \scripts/\ directory to support test infrastructure improvements. \ingest\_perf\_test\_result.py\ and \ingest\_regress\_test\_result-new-format.py\ now store performance and regression test results in a database for visualization in Grafana and Allure reports. \benchmark\_durations.py\ fetches benchmark durations from the database to optimize test distribution via the pytest-split plugin. \comment-test-report.js\ parses Allure reports to post test summaries to GitHub PRs. Additional utility scripts include \coverage\ for Rust code coverage using LLVM tools, \download\_basebackup.py\ for disaster recovery, \force\_layer\_download.py\ for managing remote layer downloads, and \proxy\_bench\_results\_ingest.py\ for collecting Prometheus metrics during proxy benchmarks.

scripts · high confidence

The \libs/utils/src\ crate now provides a centralized set of utilities for the storage system. It introduces a new JWT authentication module (\auth.rs\) that enforces EdDSA for storage tokens and defines granular scopes (Tenant, PageServerApi, SafekeeperData, etc.) for access control. It adds crash-safe file operations (\crashsafe.rs\), including \durable\_rename\ and atomic overwrites, to ensure data integrity during writes. Additionally, it provides a robust asynchronous retry mechanism with cancellation support (\backoff.rs\) and a circuit breaker (\circuit\_breaker.rs\) to handle transient failures in remote storage and network calls gracefully.

libs/utils/src · high confidence

A new client library has been added to the storage\_controller crate, exposing a public module that provides an HTTP client for communicating with the storage controller. This client handles base URL configuration, optional JWT-based authentication via Bearer tokens, and generic JSON request/response serialization for management API calls.

_storage\controller/client · high confidence

Added a new \EndpointHttpClient\ class in \test\_runner/fixtures/endpoint/http.py\ that provides a structured interface for interacting with the compute control service's HTTP API. This fixture supports bearer token authentication and exposes methods for managing the Local File Cache (LFC) prewarm and offload operations, promoting replicas, querying database schemas and metrics, and installing extensions or setting role grants, enabling tests to programmatically control and verify compute endpoint behavior.

_test\runner/fixtures/endpoint · high confidence

The test runner now includes a new README documenting the test organization, execution via the pytest wrapper, and configuration for real S3 remote storage (including AWS profile support). A new conftest.py centralizes pytest plugin loading for fixtures. A new test\_broken.py allows developers to verify fixture cleanup behavior when tests fail. Additionally, a new websocket\_tunnel.py utility enables manual and automated testing of WebSocket proxying by bridging TCP connections to WebSocket URLs, supporting the proxy's WebSocket tunneling capabilities.

_test\runner · high confidence

The pageserver now includes a new tiered compaction implementation in the \pageserver/compaction\ module, designed to reduce read amplification by reshuffling WAL records across multiple levels of delta and image layers. This change introduces a new abstraction layer (\interface.rs\) that decouples the compaction logic from the storage backend, allowing for pluggable implementations. To aid in testing and visualization, a new CLI simulator (\compaction-simulator.rs\) and in-memory mock timeline (\simulator.rs\) have been added, enabling developers to run compaction workloads (uniform or hot/cold distributions) and generate statistical reports and HTML animations of the compaction history.

pageserver/compaction/src · high confidence

The pageserver executable entry point (pageserver/src/bin/pageserver.rs) is established, initializing the server with configurable work directories, logging, OpenTelemetry tracing, and Sentry error reporting, while integrating jemalloc for memory management and supporting features like basebackup caching and tenant management. A new test helper binary (test\_helper\_slow\_client\_reads.rs) is added to simulate slow client reads by filling the page stream pipe, aiding in testing backpressure and timeout behaviors.

pageserver/src/bin · high confidence

The pageserver now exposes its management interface over HTTP, replacing the previous management protocol. This change introduces a new \pageserver/src/http\ module containing the router, request handlers, and an OpenAPI specification (\openapi\_spec.yml\). Users can now interact with the pageserver using standard HTTP endpoints for operations such as tenant and timeline management, configuration, and status checks, with authentication handled via JWT and error responses standardized to HTTP status codes.

pageserver/src/http · high confidence

The \libs/postgres\_ffi\ crate has been initialized with core infrastructure for interacting with PostgreSQL file formats. This includes a \build.rs\ script that uses bindgen to auto-generate Rust bindings for PostgreSQL v14, v15, v16, and v17 headers, a \bindgen\_deps.h\ file defining the C headers to wrap, and a sample \pg\_hba.conf\ for local development. A README has also been added to document the module's purpose and supported versions.

_libs/postgres\ffi · high confidence

The WalProposer library is now accessible from Rust code through automatically generated bindings. A new build script (build.rs) compiles the C library (libwalproposer, pgport, pgcommon) and uses bindgen to create Rust wrappers for key types and functions like WalProposerCreate, WalProposerStart, and WalProposerBroadcast. This enables Rust components to interact directly with the WalProposer logic, supporting the broader effort to migrate WalProposer functionality to Rust.

libs/walproposer · high confidence

The safekeeper now exposes a structured HTTP management API, defined by a new OpenAPI specification and implemented in the \safekeeper/src/http\ module. This includes endpoints for checking status, managing tenants and timelines (create, delete, copy, list, and get status), viewing filesystem utilization, and accessing CPU/heap profiling data. The implementation supports both HTTP and HTTPS listeners, integrates with the existing authentication system, and provides a formal contract for safekeeper control operations.

safekeeper/src/http · high confidence

The storage controller now includes its own database migration files, establishing the initial schema with \tenant\_shards\ and \nodes\ tables, and applying subsequent updates to allow null values for generation fields, rename placement policy formats, and add a scheduling policy column.

(repo-wide) · high confidence

The proxy now captures detailed connection request data—including session identifiers, authentication methods, user agents, cold-start status, and latency metrics—into structured Parquet files which are automatically uploaded to an S3-compatible remote storage bucket. This replaces ad-hoc logging with a durable, columnar audit trail for observability and analytics, configurable via new CLI arguments for storage location, file size thresholds, and compression settings.

proxy/src/context · high confidence

The proxy now includes a vendored copy of the \tokio-postgres\ library (as \tokio-postgres2\) to manage direct asynchronous connections to PostgreSQL backends. This addition introduces a complete client implementation featuring connection configuration, TLS negotiation, SASL authentication, and a codec for parsing PostgreSQL wire protocol messages. It also provides query execution, prepared statement handling, and a cancellation token mechanism, enabling the proxy to establish and maintain database sessions with full protocol support.

libs/proxy/tokio-postgres2/src · high confidence

The proxy now includes a vendored copy of the \postgres-protocol2\ crate (a subset of \rust-postgres\) to handle low-level Postgres communication. This adds support for SCRAM-SHA-256/SCRAM-SHA-256-PLUS authentication, SQL literal and identifier escaping, and the serialization/deserialization of Postgres message types (such as data rows, error responses, and notifications) and binary value types (including HSTORE and arrays).

libs/proxy/postgres-protocol2 · high confidence

The \pagectl\ utility now includes several new subcommands to aid in debugging and managing pageserver data. You can download specific objects from remote storage (S3, Azure, or local) using \download-remote-object\, visualize layer arrangements in a timeline with \draw-timeline\, and inspect or search index parts via \index-part dump\ and \index-part search\. Additional tools include \layer list\ and \layer dump\ for browsing and inspecting layer files, \key describe\ for analyzing key properties and sharding placement, \analyze-layer-map\ for evaluating delta layer holes, and \page-trace\ for parsing and summarizing page trace events.

pageserver/ctl · high confidence

The safekeeper HTTP client logic has been moved from the pageserver client into a new, dedicated \safekeeper/client\ crate. This change provides a standalone interface for communicating with safekeeper management APIs, including endpoints for timeline creation, deletion, status checks, utilization metrics, and membership switching, decoupling safekeeper-specific HTTP interactions from the pageserver.

safekeeper/client · high confidence

The HTTP utility functions previously scattered across the codebase have been consolidated into a new \libs/http-utils\ crate. This change introduces a shared, reusable implementation for HTTP server endpoints, including request handling with tracing spans, JSON parsing, error mapping to HTTP status codes, and TLS certificate reloading with metrics. It also provides a custom HTTP/HTTPS server implementation built on \hyper0\ and \routerify\, replacing the previous ad-hoc server setups in components like the pageserver and safekeeper management APIs.

libs/http-utils · high confidence

The safekeeper's public API types, including membership configuration, server information, and timeline status models, have been extracted into a new \libs/safekeeper\_api\ crate. This change centralizes shared definitions such as \SafekeeperGeneration\, \Configuration\, and various status structs, making them reusable across different components like the storage controller and safekeeper itself without internal coupling.

_libs/safekeeper\api · high confidence

The \libs/pageserver\_api\ crate has been introduced to centralize shared API definitions, configuration, and protocol types used by the pageserver and storage controller. This library now exposes the \ConfigToml\ struct for \pageserver.toml\ deserialization (silently ignoring unknown fields to support feature-flag rollouts), the \NodeMetadata\ struct for node registration, and the \controller\_api\ module containing request/response types for the storage controller's \/control/v1\ endpoints (such as \TenantCreateRequest\ and \NodeRegisterRequest\). It also includes the \pagestream\_api\ module defining the V3 pagestream protocol messages, the \key\ and \keyspace\ modules for storage key and sharded range logic, and the \models\ module defining \TenantState\ and \TenantConfig\.

_libs/pageserver\api/src · high confidence

The standalone binary entry points for the proxy, pg\_sni\_router, and local\_proxy have been relocated from the top-level bin directory into the proxy library crate (proxy/src/bin). These files now serve as thin wrappers that initialize the global allocator (jemalloc) and invoke the corresponding run functions defined within the library, centralizing the binary logic with the rest of the proxy implementation.

proxy/src/bin, proxy/src/binary · high confidence

The WAL receiver logic in the pageserver has been restructured into two new modules: \connection\_manager.rs\ and \walreceiver\_connection.rs\. The connection manager now handles the lifecycle of connections to safekeepers, using the storage broker to discover the best candidate and managing reconnection logic, while the connection handler focuses on the actual PostgreSQL replication stream and WAL ingestion. This separation isolates the connection discovery and management state from the low-level streaming and decoding tasks, improving modularity and making the code easier to maintain.

pageserver/src/tenant/timeline/walreceiver · high confidence

The authentication logic in the proxy has been reorganized into distinct, dedicated modules (classic, console\_redirect, hacks, jwt, and local) to improve code structure and maintainability. This change introduces a new \Backend\ enum that explicitly selects between the \ControlPlane\ (cloud) and \Local\ authentication flows, replacing the previous monolithic implementation. Users benefit from a cleaner separation of concerns, which supports future enhancements like improved JWT handling and local proxy configurations without impacting the core authentication path.

proxy/src/auth/backend · high confidence

The \control\_plane/src\ module has been restructured from a single monolithic \lib.rs\ into distinct, dedicated modules for managing specific infrastructure components. This change introduces separate modules for \background\_process\ (handling process lifecycle and PID files), \broker\ (managing the storage broker), \endpoint\ (managing compute endpoints), \endpoint\_storage\ (managing local object storage), \pageserver\ (managing pageserver nodes), \safekeeper\ (managing safekeeper nodes), and \local\_env\ (loading and validating the \.neon/config\ file). This modularization improves code organization and maintainability for the local development and testing environment.

_control\plane/src · high confidence

The remote timeline client logic has been reorganized into distinct modules to improve maintainability and separation of concerns. The \download.rs\ module now encapsulates layer file retrieval, including retry logic, temporary file handling, and directory fsync for durability. The \index.rs\ module manages the in-memory representation and serialization of \IndexPart\, supporting version 15 with fields for GC-compaction state, lineage, and auxiliary file policies. The \manifest.rs\ module handles tenant-wide metadata persistence, specifically the \TenantManifest\ (version 2) which tracks stripe size and offloaded timelines. Finally, \upload.rs\ centralizes all remote storage upload operations, including index parts, tenant manifests, layer files, and initdb archives, with specific handling for Azure Blob metadata and error resilience.

_pageserver/src/tenant/remote\_timeline\client · high confidence

The test suite's fixture library has been reorganized from a single monolithic module into a structured package under \test\_runner/fixtures\. This change introduces dedicated modules for specific concerns: \auth\_tokens.py\ now handles JWT token generation using the EdDSA algorithm, \common\_types.py\ provides strongly-typed Python representations for LSNs and Neon IDs, \benchmark\_fixture.py\ centralizes performance measurement logic, and \compare\_fixtures.py\ standardizes baseline comparisons. This modularization improves code maintainability and type safety for test authors.

_test\runner/fixtures · high confidence

A new workspace\_hack crate has been introduced to consolidate and manage build dependencies across the workspace. This change utilizes cargo-hakari to generate the dependency list, ensuring that build dependencies are explicitly tracked and preventing potential build issues caused by missing or conflicting dependencies.

_workspace\hack · high confidence

A new database migration adds an index on the \tenant\_id\ column of the \tenant\_shards\ table. This optimization is intended to support the storage controller's ability to run timeline CRUD operations concurrently with reconciliation by speeding up lookups based on tenant ID.

_storage\_controller/migrations/2024-08-23-170149\_tenant\_id\index · medium confidence

The pageserver now persists and uploads consumption metrics in Newline Delimited JSON (NDJSON) format instead of the previous JSON structure. This change updates the disk cache to read and write metrics using the new format, and modifies the upload logic to compress metrics into gzipped NDJSON files when sending them to remote storage, ensuring compatibility with downstream billing and monitoring systems that expect this schema.

_pageserver/src/consumption\metrics · high confidence

The storage controller's database schema is updated via a new migration to add a \sk\_set\_notified\_generation\ column to the \timelines\ table. This integer column, defaulting to 1 and not nullable, tracks the generation of safekeeper notifications, supporting the graceful completion of safekeeper migration processes.

_storage\_controller/migrations/2025-07-08-114340\_sk\_set\_notified\generation · high confidence

The storage controller's database schema now includes two new tables, \hadron\_safekeepers\ and \hadron\_timeline\_safekeepers\, to track Safe Keeper nodes and their associations with timelines. This migration introduces the necessary storage structure for the Hadron cluster coordination, allowing the system to register Safe Keeper node IDs, listen addresses, and map them to specific timelines.

_storage\_controller/migrations/2025-07-17-000001\_hadron\safekeepers · high confidence

The storage controller's database schema now includes two new tables: \timelines\, which tracks tenant and timeline metadata including LSN, generation, and safekeeper sets, and \safekeeper\_timeline\_pending\_ops\, which records pending operations associated with specific safekeepers. This migration introduces the necessary persistence layer to support the creation and deletion workflows for timelines managed by the storage controller.

_storage\_controller/migrations/2025-02-14-160526\_safekeeper\timelines · high confidence

The storage controller's database schema now includes a new \timeline\_imports\ table to track the status of timeline imports across shards. This table stores the tenant ID, timeline ID, and a JSONB field for shard statuses, with a composite primary key on tenant and timeline IDs, enabling the system to coordinate import operations more effectively.

_storage\_controller/migrations/2025-03-18-103700\_timeline\imports · high confidence

The storage controller's database migration now creates a \safekeepers\ table to store metadata for safekeeper nodes. This table records each node's unique ID, region, version, host, port, HTTP port, availability zone, and active status, enabling the controller to manage and upsert safekeeper records during deployment.

_storage\_controller/migrations/2024-08-23-102952\safekeepers · high confidence

The storage controller's database schema now includes a \listen\_https\_port\ column in the \nodes\ table, enabling the system to store and manage HTTPS port configurations for pageservers. This change supports the migration to using HTTPS for communication with pageservers.

_storage\_controller/migrations/2025-02-11-144848\_pageserver\_use\https · high confidence

The storage controller's database schema now includes a new \controllers\ table to track leadership state. This table stores the node address and the timestamp when leadership started, using a composite primary key on both fields. This change enables the system to maintain accurate records of which node currently holds leadership, supporting graceful leadership transfer mechanisms.

_storage\_controller/migrations/2024-07-26-140924\_create\leader · high confidence

The compute\_ctl HTTP middleware now strictly validates incoming JWT tokens by checking the 'aud' (audience) claim for Admin-scoped requests and verifying the 'compute\_id' for other scopes. This change introduces a new authorization middleware that supports both EdDSA and RS256 algorithms (specifically for Hadron instances) and ensures that only tokens with the correct audience ('compute') and matching compute ID are accepted, returning 401 or 403 errors for invalid claims. Additionally, a request ID middleware is added to generate UUIDs if not provided by the caller.

_compute\tools/src/http/middleware · high confidence

The pageserver now implements a retry mechanism with exponential backoff when launching the tokio-epoll-uring io engine fails due to insufficient locked memory (ENOMEM). This change specifically targets older Linux kernels (e.g., 5.10 series) where io\_uring creation is more prone to memory allocation failures. If a launch fails, the system logs detailed process statistics (such as memory limits and RSS) to aid debugging and automatically retries the initialization, preventing immediate service disruption that would otherwise occur from a panic or fatal error.

_pageserver/src/virtual\_file/io\engine · high confidence

The walproposer component in libs/walproposer/src has been rewritten in Rust, replacing the previous C implementation. This change introduces a new C-Rust shim (api\_bindings.rs) that implements the C walproposer API, allowing the Rust code to interact with the existing Postgres safekeeper infrastructure. The core logic is now defined in walproposer.rs, which provides a high-level Rust wrapper for the C API, including traits for API implementation and configuration structures for tenant/timeline IDs and safekeeper lists. This shift enables better memory safety and maintainability for the WAL proposer functionality.

libs/walproposer/src · high confidence

The \libs/pq\_proto\ crate now provides a custom \Framed\ abstraction for reading and writing Postgres wire protocol messages, replacing previous inline implementations. This change introduces cancellation-safe, separate read/write buffers (\FramedReader\ and \FramedWriter\) that avoid the allocation overhead of \tokio::io::split\ while supporting startup packets, standard messages, and connection error handling. It also includes robust parsing for \StartupMessageParams\, including proper handling of escaped command-line options, and defines core message types like \FeMessage\ and \CancelKeyData\ for protocol interaction.

_libs/pq\proto · high confidence

The pageserver now uses a new \owned\_buffers\_io\ subsystem for writing, leveraging \tokio-epoll-uring\ to perform direct IO. This change introduces aligned buffer handling, a \BufferedWriter\ that batches small writes into larger, aligned flushes, and a background flush task to improve write performance and reliability. The implementation includes specific handling for write errors (retrying indefinitely), cancellation sensitivity, and various shutdown modes (drop, zero-pad, pad-and-truncate) to manage pending data correctly.

_pageserver/src/virtual\_file/owned\_buffers\io · high confidence

The pageserver client now uses a new gRPC-based implementation that replaces the previous transport. This client features dedicated connection and stream pools to efficiently reuse resources across concurrent callers, avoiding the overhead of establishing new TCP connections for every request. It includes automatic retry logic with exponential backoff for transient errors, handles sharded tenants by routing requests to the correct shards, and splits large GetPage batches across shards to improve throughput. Bulk requests are routed to a separate pool to prevent head-of-line blocking, and idle resources are reaped to free up memory.

_pageserver/client\grpc · high confidence

The pageserver's write path now uses a dedicated background task to handle flushing dirty buffers to disk. This change introduces a \FlushHandle\ that communicates with the background task via a duplex channel, allowing the write path to submit buffers for flushing and receive recycled buffers back. This mechanism provides backpressure when the flush task cannot keep up and ensures proper cleanup and shutdown handling for the buffered writer.

_pageserver/src/virtual\_file/owned\_buffers\io/write · high confidence

The layer map now uses persistent (immutable) Red-Black trees to track which storage layers cover specific key ranges at different LSNs. This change enables efficient, versioned queries for layer visibility without the performance penalty of cloning large data structures, supporting the new range layer map search and visibility calculation logic.

_pageserver/src/tenant/layer\map · high confidence

The pageserver now includes a native Rust implementation for applying specific WAL redo operations (visibility map truncation/clearing and CLOG/multixact status updates) directly, bypassing the external Postgres process for these tasks. This new \apply\_neon\ module handles \NeonWalRecord\ variants in-process, while still delegating standard Postgres WAL records to the existing external \walredo\ process via the new \process.rs\ IPC protocol. This change improves performance and reliability for these common internal operations by reducing inter-process communication overhead.

pageserver/src/walredo · high confidence

This change introduces new configuration files in the compute environment to support metrics collection and connection pooling. It adds a README explaining how to add sql\_exporter metrics, defines the neon\_collector and neon\_collector\_autoscaling configurations in Jsonnet to export a wide range of Postgres and Neon-specific metrics (including LFC stats, backpressure, and replication delay), and sets up pgbouncer with a default application\_name and disabled connection logging to reduce log noise.

compute/etc · high confidence

The pageserver now uses a new, multi-stage deletion queue to manage remote object removal. This system persists deletion intents to local disk via a ListWriter, ensuring they survive restarts, and introduces a Validator stage that checks tenant generations against the storage controller before execution to prevent split-brain data loss. Finally, a Deleter stage batches and executes the validated deletions with retry logic, improving reliability and efficiency of remote storage cleanup.

_pageserver/src/deletion\queue · high confidence

The pageserver introduces dedicated modules for timeline analysis, compaction, deletion, ancestor detachment, eviction, and request handling. The new analysis module provides performance insights by calculating read amplification across key ranges. A new compaction implementation (replacing legacy logic in timeline.rs) supports GC-compaction, shard ancestor compaction, and automatic triggering with detailed statistics. Timeline deletion is now a resumable flow that safely handles local and remote cleanup. Ancestor detachment allows timelines to be split from their parents with layer rewriting and tombstone generation. The eviction task is refactored to support configurable policies and imitation-only modes. Finally, a new handle/cache system optimizes request dispatching by reducing lock contention and gate-enter overhead for page service connections.

pageserver/src/tenant/timeline · high confidence

The storage controller's database schema now includes a 'lifecycle' column on the 'nodes' table, allowing the system to track the operational state of nodes (e.g., active, deleted). This change supports handling flaky node scenarios by introducing deletion tombstones, enabling the controller to distinguish between active nodes and those marked for removal without immediate physical deletion.

_storage\_controller/migrations/2025-06-01-201442\_add\_lifecycle\_to\nodes · high confidence

The storage controller database schema has been updated to enforce that the \availability\_zone\_id\ field in the \nodes\ table is not null. This change ensures that every pageserver record must have an assigned availability zone, preventing the creation or persistence of nodes without this required metadata.

_storage\_controller/migrations/2024-08-28-150530\_pageserver\_az\_not\null · high confidence

The proxy now uses a dedicated thread pool to handle SCRAM password hashing asynchronously, preventing authentication requests from blocking the main event loop. To further improve performance for active users, the system caches the suffix of the PBKDF2 hash result in memory, allowing subsequent logins to skip most of the expensive hashing work. Additionally, a Count-Min Sketch is used to estimate per-endpoint hashing rates, enabling the proxy to dynamically throttle or skip hashing jobs for endpoints that are generating excessive load, thereby protecting the system from resource exhaustion.

proxy/src/scram · high confidence

The \postgres\_initdb\ library now explicitly supports PostgreSQL 17, introducing a \PgMajorVersion\ enum to replace ad-hoc version handling. This change ensures that \initdb\ runs with deterministic locale settings by passing \--locale-provider builtin\ for PostgreSQL 17 (which uses a safer, consistent builtin provider) and \libc\ for versions 15 and 16, while eliminating environment-dependent implicit defaults. The \postgres\_versioninfo\ library provides the underlying type-safe version representation used by this logic.

_libs/postgres\initdb · high confidence

The \libs/postgres\_ffi\ crate has been reorganized to support PostgreSQL versions 14, 15, 16, and 17 simultaneously. The library now uses a macro-based dispatch system to generate version-specific modules (e.g., \v14\, \v15\, \v16\, \v17\) that contain their own bindings, constants, and utilities. This change introduces version-specific constant files (\pg\_constants\_v14.rs\ through \pg\_constants\_v17.rs\) to handle differences in WAL record formats and data structures across versions. Additionally, the crate now includes a WAL generator for creating test/benchmark data, a dedicated benchmark suite for the WAL decoder, and utilities for parsing PostgreSQL control files and relation filenames.

_libs/postgres\ffi/src · high confidence

The proxy's TLS handling has been rewritten to use the \rustls\ library instead of the previous OpenSSL-based implementation. This change introduces pre-loading of native system certificates and internal CA certificates for compute connections, ensuring reliable TLS verification. It also implements proper channel binding (RFC 5929) for PostgreSQL connections by hashing server certificates, and supports separate TLS configurations for HTTP and PostgreSQL ALPN protocols on the server side.

proxy/src/tls · high confidence

The proxy source code has been significantly restructured to improve reliability and observability. A new intrusive linked-list-based batch processing system (batch.rs) now handles cancellation requests, allowing for efficient, allocation-free queuing with direct cancellation support. Cancellation logic has been moved to a dedicated module (cancellation.rs) that uses Redis pipelines for batched operations. The proxy now features a new console redirect proxy flow (console\_redirect\_proxy.rs) and a dedicated Compute Control API (compute\_ctl/mod.rs) for managing extensions and role grants. Observability is enhanced with a new jemalloc metric recorder (jemalloc.rs) and a comprehensive metrics module (metrics.rs) using the 'measured' library. Configuration is centralized in config.rs, and error handling is standardized via a new error.rs module defining specific error kinds (e.g., Quota, RateLimit) and a UserFacingError trait to prevent sensitive data leaks.

proxy/src · high confidence

The proxy's rate-limiting logic has been restructured into distinct, reusable modules. A new leaky-bucket implementation (leaky\_bucket.rs) now handles per-endpoint connection rate limiting using a configurable RPS and burst capacity, backed by a shard-based map for performance. Additionally, a new dynamic concurrency limiter (limit\_algorithm.rs) introduces an adaptive algorithm (AIMD) alongside a fixed-limit mode, allowing the proxy to automatically adjust concurrency limits based on observed latency and overload outcomes. The existing bucket-based rate limiter (limiter.rs) is retained for specific use cases like wake-compute limits, utilizing multiple time-window buckets. This change provides more granular control and adaptability in how the proxy manages traffic and prevents overload.

_proxy/src/rate\limiter · high confidence

The proxy's Redis connection layer has been refactored to support dynamic credential rotation via AWS IAM, specifically for ElastiCache clusters. A new \CredentialsProvider\ implementation handles AWS credential acquisition (supporting environment variables, profiles, web identity tokens, and IMDSv2) and generates signed SigV4 authentication tokens, which are automatically refreshed by a background task to maintain the connection. This replaces static credential usage, ensuring the proxy can reconnect and authenticate securely without manual intervention when tokens expire.

proxy/src/redis · high confidence

The proxy's caching layer has been restructured to replace the previous \dashmap\ and \TimedLru\ implementations with \moka\ for TTL-based caches and \clashmap\ for the project-info cache. This change introduces a new \common.rs\ module that defines a generic \Cache\ trait, a \Cached\ wrapper for automatic invalidation, and a \CplaneExpiry\ strategy that allows errors to expire early based on retry-after headers. The new \ProjectInfoCache\ and \NodeInfoCache\ implementations integrate these components, ensuring that cache hits, misses, and evictions are now tracked via specific metrics (CacheKind) for better observability.

proxy/src/cache · high confidence

The proxy's compute connection handling has been restructured to separate connection establishment from authentication. New modules (\proxy/src/compute/mod.rs\ and \tls.rs\) introduce dedicated types like \AuthInfo\ and \ConnectInfo\ to manage configuration, including explicit support for disabling channel binding in console-redirect scenarios and handling SCRAM-SHA-256 keys. This change also standardizes TLS negotiation and error reporting, ensuring that connection errors are correctly classified as user-facing or internal.

proxy/src/compute · medium confidence

The compute migration scripts in \compute\_tools/src/migrations\ have been restructured to start at 0001 and now accept a \privileged\_role\_name\ parameter instead of hardcoding the role name. This allows the migration logic—such as granting \BYPASSRLS\, \pg\_monitor\, and replication-related privileges—to be applied to a configurable role. Additionally, comprehensive SQL-based tests have been added under \compute\_tools/src/migrations/tests\ to verify that the specified privileged role (e.g., \neon\_superuser\) correctly inherits these permissions and that non-privileged roles have \BYPASSRLS\ and replication rights revoked, ensuring the security posture defined in the migrations is enforced.

_compute\tools/src/migrations · high confidence

The proxy's control plane module has been restructured to improve error clarity and security. Error messages previously referring to the "Console" now correctly identify the "Control plane," and a unified error interface standardizes how API failures are reported and retried. Additionally, the module now enforces stricter access controls by validating IP addresses and VPC endpoint IDs, and it supports per-endpoint rate limiting for connection attempts.

_proxy/src/control\plane · high confidence

The storage layer implementation in \pageserver/src/tenant/storage\_layer\ has been completely rewritten. This change introduces new core types such as \BatchLayerWriter\ and \SplitImageLayerWriter\ to support atomic layer creation and splitting, replaces the previous \LayerDescriptor\ with \PersistentLayerDesc\ for unified layer identification, and adds a \FilterIterator\ to enable key-space filtering during merge operations. These updates form the foundation for the new compaction abstraction and improved vectored read planning.

_pageserver/src/tenant/storage\layer · high confidence

The authentication logic in the proxy has been restructured into dedicated modules (\credentials.rs\, \flow.rs\, \password\_hack.rs\) to improve clarity and maintainability. This change introduces a new \ComputeUserInfoMaybeEndpoint\ struct to handle user credentials and endpoint identification, supporting both SNI-based routing and a legacy 'password hack' method for clients that do not support SNI. The 'password hack' payload parsing has been updated to split on the first occurrence of either \;\ or \$\, ensuring that passwords containing these characters are not truncated. Additionally, the SCRAM authentication flow now supports channel binding, and the system enforces stricter validation for project names and IP allowlists.

proxy/src/auth · high confidence

The proxy's connection logic has been restructured to clearly separate the steps of connecting to a compute node and authenticating with it. This change introduces a new retry mechanism that specifically handles stale cache entries during the authentication phase, ensuring that if a connection succeeds but authentication fails due to a stale cache, the proxy will retry with a fresh node lookup. Additionally, the proxy now exposes the session ID to clients via greeting messages and provides detailed latency metrics (control plane, client, compute, and retry) to test probes for better observability.

proxy/src/proxy · high confidence

The database schema for the storage controller has been updated to remove the 'active' boolean column from the 'safekeepers' table. This change simplifies the storage controller's data model by eliminating a field that is no longer required for tracking safekeeper status.

_storage\_controller/migrations/2025-01-09-160454\_safekeepers\_remove\active · high confidence

The safekeeper executable now exposes a detailed command-line interface for configuring WAL service endpoints (including a new tenant-scoped listener), HTTP/HTTPS management APIs, remote storage offloading, and independent JWT authentication keys for each service. It also introduces flags for controlling disk usage limits, WAL backup behavior, and logging format, allowing operators to fine-tune the safekeeper's storage and network behavior directly at startup.

safekeeper/src/bin · high confidence

The safekeeper source code has been reorganized into distinct modules for authentication, broker communication, control file persistence, and timeline copying, while introducing a new \hadron\ module that enables the safekeeper to register with the Hadron Cluster Coordinator (HCC) via HTTP using JWT tokens from the \HCC\_AUTH\_TOKEN\ environment variable. This change also includes the implementation of a new \check\_permission\ function in \auth.rs\ to enforce tenant-scoped JWT access for management APIs and updates the control file format (v10) to support generation-aware state and membership configuration.

safekeeper/src · high confidence

The default scheduling policy for new safekeepers has been updated from 'pause' to 'activating'. This database migration ensures that newly created safekeeper records are automatically set to an active state, removing the previous default behavior of starting in a paused state.

_storage\_controller/migrations/2025-07-02-170751\_safekeeper\_default\_no\pause · high confidence

The storage controller database migration updates the safekeepers table to rename the 'disabled' scheduling policy value to 'pause'. Existing records with the 'disabled' policy are migrated to 'pause', and the default value for the scheduling\_policy column is changed from 'disabled' to 'pause'.

_storage\_controller/migrations/2025-01-15-181207\_safekeepers\_disabled\_to\pause · high confidence

The storage controller's database schema now includes an https\_port column in the safekeepers table, enabling the system to store and manage the HTTPS port configuration for safekeepers. This change supports the migration to using the HTTPS API for safekeeper communication.

_storage\_controller/migrations/2025-02-28-141741\_safekeeper\_use\https · high confidence

The storage controller's database schema has been updated to include a new \scheduling\_policy\ column in the \safekeepers\ table. This column is a non-nullable string with a default value of 'disabled', allowing the system to manage how safekeepers are scheduled without requiring immediate configuration changes for existing deployments.

_storage\_controller/migrations/2024-12-12-212515\_safekeepers\_scheduling\policy · high confidence

The HTTP layer now uses custom extractors for JSON bodies, path parameters, and query strings that ensure all parsing failures return consistent JSON-formatted API errors instead of generic Axum rejection responses. Additionally, a new RequestId extractor automatically captures the X-Request-Id header, enabling better request tracing and debugging for API consumers.

_compute\tools/src/http/extract · high confidence

Developers now have consistent coding standards and build automation for the Neon extensions in the pgxn directory. New .editorconfig and .dir-locals.el files enforce uniform indentation and formatting across C, XML, and Perl files, while a new Makefile provides structured targets to build and install the neon, neon\_rmgr, neon\_walredo, neon\_utils, and neon\_test\_utils extensions, supporting both VPATH builds and Docker-based compute node setups.

pgxn · high confidence

The storage controller's database schema has been updated to support gRPC communication with pageservers. A new migration adds the \listen\_grpc\_addr\ (string) and \listen\_grpc\_port\ (integer) columns to the \nodes\ table, allowing the system to store and manage the network endpoints required for gRPC-based interactions.

_storage\_controller/migrations/2025-06-17-082247\_pageserver\_grpc\addr · high confidence

The storage controller database schema now includes an availability\_zone\_id column on the nodes table, allowing the system to record and track the availability zone of each pageserver. This change supports better placement and scheduling decisions by making zone information available to the controller.

_storage\_controller/migrations/2024-08-27-184400\_pageserver\az · high confidence

The storage controller's database schema now includes a \preferred\_az\_id\ column in the \tenant\_shards\ table, allowing the system to record and track the preferred availability zone for each tenant shard.

_storage\_controller/migrations/2024-09-05-104500\_tenant\_shard\_preferred\az · high confidence

The build-tools Dockerfile has been moved to a dedicated directory and now pins the Debian base image (bookworm-slim and bullseye-slim) to specific SHA256 digests for reproducible builds. It includes a patch for pgcopydb v0.17 to fix potential null-pointer issues and adjust session timeouts, and adds several new tools: sql\_exporter v0.17.3, protobuf-compiler v25.1, s5cmd v2.3.0, LLVM 20, and Node.js 24.

build-tools · high confidence

This change introduces several patches to the compute environment to improve test stability and configuration. It adds patches for the \anon\ v2 extension to grant permissions to a privileged role and refactors its data loading function, and updates \pg\_duckdb\ to use a renamed library to avoid conflicts. Regression test suites for Postgres 16 and 17 are patched to handle password placeholders and file copy commands, while \pg\_repack\ tests are updated to disable autovacuum and handle non-superuser checks. Additionally, patches for \pg\_hint\_plan\ and \pg\_graphql\ adjust expected outputs and permissions for Neon's environment, and \onnxruntime\ dependencies are switched to a more reliable mirror.

compute/patches · high confidence

The proxy now includes a vendored subset of the rust-postgres library, specifically introducing the error module at \libs/proxy/tokio-postgres2/src/error\. This adds structured support for parsing PostgreSQL error responses via the \DbError\ type and defines standard SQLSTATE error codes (such as connection failures and syntax errors) in \sqlstate.rs\, enabling the proxy to classify and handle database errors more precisely.

libs/proxy/tokio-postgres2/src/error · high confidence

The proxy now includes a vendored copy of the \postgres-types2\ crate (located in \libs/proxy/postgres-types2\) to manage conversions between Rust and Postgres types. This addition introduces core type definitions, including the \Type\ struct and \Kind\ enum, alongside generated mappings for built-in Postgres types (such as \Int4\, \Text\, \Jsonb\, and various arrays) and error types like \WrongType\ and \WasNull\. This change supports the proxy's internal type resolution and serialization logic by providing a self-contained implementation of the type system previously relied upon from external dependencies.

libs/proxy/postgres-types2 · high confidence

The pageserver's virtual file layer now supports pluggable IO engines, allowing the system to switch between standard filesystem operations and Linux-specific \tokio-epoll-uring\ for improved performance. This change introduces a global, live-reconfigurable \IoEngine\ selection mechanism that dispatches read, write, and metadata operations to the active backend. It also adds support for Direct I/O via the \O\_DIRECT\ flag, including runtime alignment validation to prevent data corruption on platforms where direct I/O is not natively supported (such as macOS), ensuring safer and more efficient disk access patterns for the write path.

_pageserver/src/virtual\file · high confidence

The \neon\_local\ executable has been relocated to \control\_plane/src/bin\ and updated to reflect the new storage controller architecture. The CLI now manages the \storage\_controller\ and \storage\_broker\ as first-class subcommands (with aliases \storage\_controller\ and \storage\_broker\ for backward compatibility), replacing the previous direct pageserver/safekeeper management model. It defaults to PostgreSQL 17, supports sharding configuration (shard count, stripe size, placement policy) during tenant creation, and integrates with the new endpoint storage system for managing compute endpoints. The initialization process now uses \pageserver.toml\ directly and supports a \--force\ mode to overwrite existing directories.

_control\plane/src/bin · high confidence

The Neon compute configuration now validates the Postgres version against a supported range (14–17). If an unsupported version is provided, the configuration process will fail with a clear error message, preventing deployment with incompatible Postgres versions.

compute/jsonnet · high confidence

Resolves a data corruption risk where concurrent import operations could fail due to a race condition during merge, ensuring reliable timeline imports even under concurrent load.

pageserver/src · high confidence

Add Neon-specific SQL regression tests; Add PostgreSQL extension test infrastructure and regression suites; Add multi-language PostgreSQL client integration tests; Add pagebench-based performance regression tests for the pageserver; Added Docker Compose test infrastructure for Neon images and extensions; Added TLA+ model checking tests for safekeeper consensus and reconfiguration; Added discrete event simulation framework for safekeeper and walproposer; Added discrete-event simulation tests for safekeeper crash and restart scenarios; Added integration tests for postgres\_backend with TLS support; Added integration tests for real Azure, GCS, and S3 remote storage backends; Added integration tests for remote storage listing and pagination; Added interactive debugging tool for pageserver performance testing; Added random operations stability test for Neon API; Added regression tests for cloud-based Neon instances; Added tests for binary serialization consistency; Added tests for consumption metrics collection logic; Added tests for logical replication with ClickHouse and Debezium; Added tests for proxy channel binding and MITM resistance; Added tests for tiered compaction behavior; Added unit tests for compute\_tools configuration and helper functions; Added unit tests for tenant size model calculations; Extensive regression test suite updates and stability improvements; New failpoint and test infrastructure for layer eviction and download control; New safekeeper test fixtures for HTTP API interaction and status verification; New test utility to generate deep L0 delta stacks.

The project now includes a Cargo.lock file and updated Cargo.toml workspace definitions, establishing a pinned dependency graph for the Rust codebase. Additionally, build-tools and compute directories now contain package-lock.json files, introducing npm dependencies for OpenAPI documentation generation and JSON schema validation.

(dependencies) · high confidence

The vendored PostgreSQL submodules have been updated to include the latest minor releases: version 14.18, 15.13, 16.9, and the newly added 17.5. These updates bring in upstream bug fixes and improvements for all supported PostgreSQL versions used by the platform.

vendor · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 38 → 61 (+22.9)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 100 → 91 (-9.2)
• Architecture 69 → 97 (+27.7)
• Maturity 82 → 90 (+7.6)
• Readiness 27 → 55 (+27.6)
• Security 21 → 44 (+22.7)
• Domain Modelling 100 (new)
• Event Sourcing 100 (new)

• ADR not followed: Splitting cloud console (docs/rfcs/017-console-split.md)
• All five are named 'Eviction' variants but the visible text is a repetitive decision list with no context/problem and no trade-offs for each variant (docs/rfcs/012-background-tasks.md)
• Coverage not measured — test suite did not build
• Critical CVE: [GHSA redacted] (build-tools/package-lock.json)
• Critical CVE: [GHSA redacted] (build-tools/package-lock.json)
• Critical CVE: [GHSA redacted] (build-tools/package-lock.json)
• Dimension evaluation failed
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (poetry.lock)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (build-tools/package-lock.json)
• High CVE: [GHSA redacted] (poetry.lock)
• …and 120 more

• AcceptorProposerMessage::serialize (cognitive 19) (safekeeper/src/safekeeper.rs)
• Ambiguous naming for configuration updates. ComputeNode exposes both reconfigure and signal_refresh_configuration, while Endpoint exposes reconfigure and refresh_configuration. It is unclear if these are distinct operations (e.g., immediate vs async, or config reload vs metadata update) or if the naming is inconsistent across the two main compute types.
• AzureBlobStorage::list_streaming_for_fn (cognitive 33) (libs/remote_storage/src/azure_blob.rs)
• AzureBlobStorage::list_streaming_for_fn (cyclomatic 20) (libs/remote_storage/src/azure_blob.rs)
• AzureBlobStorage::time_travel_recover (cognitive 32) (libs/remote_storage/src/azure_blob.rs)
• BackgroundTask::cleanup (cognitive 17) (pageserver/src/basebackup_cache.rs)
• BaseBackupCmd::parse (cognitive 16) (pageserver/src/page_service.rs)
• Basebackup::add_dbdir (cognitive 16) (pageserver/src/basebackup.rs)
• Basebackup::send_tarball (cognitive 40) (pageserver/src/basebackup.rs)
• Basebackup::send_tarball (cyclomatic 24) (pageserver/src/basebackup.rs)
• BatchLayerWriter::finish_with_discard_fn (cognitive 17) (pageserver/src/tenant/storage_layer/batch_split_writer.rs)
• BatchQueue::call (cognitive 29) (proxy/src/batch.rs)
• BatchedFeMessage::should_break_batch (cognitive 24) (pageserver/src/page_service.rs)
• BeMessage::write (cognitive 19) (libs/pq_proto/src/lib.rs)
• BeMessage::write (cyclomatic 37) (libs/pq_proto/src/lib.rs)
• BlobWriter::write_blob_maybe_compressed (cognitive 17) (pageserver/src/tenant/blob_io.rs)
• BlockCursor::read_blob_into_buf (cognitive 21) (pageserver/src/tenant/blob_io.rs)
• Boundary-crossing change coupling: compute.rs ↔ requests.rs (compute_tools/src/compute.rs)
• CI installs an unverified third-party binary (.github/workflows/build_and_test.yml)
• Change coupling: block_io.rs ↔ ephemeral_file.rs (pageserver/src/tenant/block_io.rs)
• …and 1078 more

• 1 commits — 1 feature/other, 0 fixes

• docs/rfcs — 1 commit

• change: docs: fix typo proccess -> process (#12940)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

neondatabase/neon was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.