Skip to content
CAI
Software that uses CAICheck a score

vectordotdev/vector

56.6

Adequate · 27 September 2026

258.6k

lines of production code

Rust

primary language

4

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Vector is a high-performance data pipeline and observability agent designed to ingest, transform, and route telemetry data across diverse protocols. It provides a comprehensive suite of sources, transforms, and sinks that handle logs, metrics, and traces, featuring robust codec support for formats like OTLP, Prometheus, and Datadog. The system emphasizes reliability through configurable buffering, schema validation, and enrichment capabilities, while offering deep internal observability via a gRPC-based API and interactive monitoring tools.

How it got here

2018–2021 — Vector 1.0 architecture and ecosystem expansion

94 changes.

This period focused on stabilizing Vector's core architecture through a comprehensive rewrite of the sink and topology subsystems, introducing a unified event schema and gRPC-based observability API. It simultaneously expanded the platform's capabilities by adding numerous new sources, sinks, and transforms, while establishing robust distribution, testing, and CI infrastructure for major operating systems and cloud providers.

2022 — buffer rewrite and codec unification

60 changes.

This period focused on a major architectural overhaul of Vector's core infrastructure, introducing a new disk-backed buffer system with zero-copy serialization and a unified codec library for standardized encoding and decoding. Concurrently, the project established a declarative configuration schema system using procedural macros to enable robust validation and documentation. These foundational changes were supported by the addition of numerous new sinks and sources, including support for OpenTelemetry, Datadog, and various cloud providers.

2023–2026 — Sink expansion and VRL ecosystem

68 changes.

This period focused on significantly expanding Vector's sink capabilities with numerous new integrations for databases, cloud services, and messaging systems, while refactoring existing sinks to a modern stream-based architecture. Concurrently, the project enhanced the Vector Remap Language (VRL) ecosystem by introducing a web playground, standalone CLI, and new functions for enrichment and metrics inspection. Underlying infrastructure improvements included consolidating internal dependencies into a unified library and implementing stricter security confinement for URI templates.

Features

Add AWS CloudWatch Metrics sink with high-resolution support

The new \aws\_cloudwatch\_metrics\ sink allows users to publish metric events to AWS CloudWatch. It supports counters, gauges, distributions, and sets, and includes a \storage\_resolution\ configuration option to enable high-resolution (1-second) metrics alongside standard (60-second) ones. The sink handles metric normalization, batching, and includes integration tests for health checks and namespace partitioning.

_src/sinks/aws\_cloudwatch\metrics · high confidence

Add AWS SNS and SQS sinks

This change introduces two new sinks, \aws\_sns\ and \aws\_sqs\, allowing you to publish observability events to AWS Simple Notification Service topics and Simple Queue Service queues. The implementation includes configuration for encoding, authentication, TLS, and request settings, as well as validation logic that enforces \message\_group\_id\ requirements for FIFO queues and topics. The code also provides healthcheck capabilities and integration tests to verify message delivery.

_src/sinks/aws\_s\s · high confidence

Add Apache Doris sink support

Introduces a new Apache Doris sink that delivers log data to an Apache Doris database using the Stream Load protocol. The implementation supports configurable endpoints, database and table routing via templates, basic authentication, TLS, compression, and custom HTTP headers for Stream Load parameters. It includes built-in health checks, retry logic for server errors and non-success Stream Load statuses, and emits metrics for loaded and filtered rows.

src/sinks/doris · high confidence

Add Keep sink for delivering log events to Keep

A new \keep\ sink has been added, allowing Vector to deliver log events to the Keep service. The sink supports configurable API key authentication, HTTP endpoint configuration, batching, request limits, and retry strategies. It encodes events into JSON payloads, handles health checks, and integrates with the standard Vector sink infrastructure for reliable event delivery.

src/sinks/keep · high confidence

Add MQTT sink for publishing telemetry data

Introduces a new MQTT sink that allows Vector to publish log events to an MQTT broker. The sink supports configurable topics (with template rendering), Quality of Service levels (AtMostOnce, AtLeastOnce, ExactlyOnce), message retention flags, and TLS settings. It includes compile-time configuration validation, client ID generation, and integration tests to verify message delivery.

src/sinks/mqtt · high confidence

Add OpenTelemetry protocol conversion library for logs, metrics, and traces

The new \lib/opentelemetry-proto\ library provides the core logic to convert incoming OpenTelemetry Protocol (OTLP) data into Vector's internal event formats. It includes implementations for parsing OTLP logs, metrics (including gauges, sums, histograms, and summaries), and traces, mapping them to Vector's log, metric, and trace event structures while preserving resource and scope metadata. The library also handles edge cases such as converting NaN values to null to prevent panics during metric ingestion.

lib/opentelemetry-proto/src · high confidence

Add PostgreSQL sink for delivering log data to a database

A new PostgreSQL sink has been added, allowing users to deliver log data directly to a PostgreSQL database. The implementation supports configurable connection endpoints, table names, and connection pool sizes, with built-in validation of connection strings at compile time. It handles event batching and serialization into JSON for efficient insertion, and includes retry logic for transient network or pool errors. Integration tests verify health checks and error handling for unknown hosts.

src/sinks/postgres · high confidence

Add WebHDFS sink for writing events to Hadoop Distributed File System

Users can now send log events to a WebHDFS cluster using the new \webhdfs\ sink. This feature allows configuring the HDFS endpoint, root directory, and file prefix (supporting templating for partitioning), while leveraging the existing OpenDAL integration for encoding, compression, and batching. The sink includes built-in health checks and validation to ensure connectivity and correct configuration before runtime.

src/sinks/webhdfs · high confidence

Add WebSocket sink for delivering observability data

A new \websocket\ sink is now available, allowing you to deliver observability event data to a WebSocket listener. The sink supports configurable encoding (defaulting to JSON) and optional end-to-end acknowledgements. It handles connection management, including automatic ping/pong keep-alive intervals and timeouts, and emits internal metrics for bytes sent and events delivered.

src/sinks/websocket · high confidence

Add configurable length-delimited framing options

Users can now configure the framing behavior for length-delimited codecs through the new \LengthDelimitedCoderOptions\ component. This allows control over the maximum frame length, the number of bytes used for the length field, the offset of the length field within the header, and whether the length field uses big-endian or little-endian byte order, enabling compatibility with various binary protocols that require specific framing configurations.

lib/codecs/src/common · high confidence

Add configuration schema support for external library types

The configuration system now supports schema generation and serialization for several external types, including \chrono::DateTime\, \chrono\_tz::Tz\, \url::Url\, \indexmap::IndexMap/Set\, \serde\_with\ duration helpers, \toml::Value\, and VRL types (\VrlRuntime\, \QueryNode\, \VrlValue\). This enables these types to be used directly in configuration files with proper schema validation and documentation generation.

lib/vector-config/src/external · high confidence

Added SysV init script for Vector

A new SysV init script has been added to the distribution, enabling users to manage the Vector service using standard init commands (start, stop, restart, reload, status) on systems that rely on the System V init system rather than systemd. The script configures Vector to run as a background daemon, manages the PID file at /var/run/vector.pid, and sources the standard function library for consistent service behavior.

distribution/init.d · high confidence

Added buffer performance benchmarking tool

A new \buffer\_perf\ example has been added to the \lib/vector-buffers\ crate, providing a CLI-based performance testing runner for buffer implementations. This tool allows users to benchmark disk-v1, disk-v2, and in-memory buffer types by configuring parameters such as total read/write record counts, batch sizes, and record size ranges, utilizing the \EventCount\ trait and new buffer topology channels for accurate performance measurement.

lib/vector-buffers/examples · high confidence

Added example code for tracing-limit rate limiting

Added two Rust example programs (\basic.rs\ and \by\_span.rs\) to the \lib/tracing-limit/examples\ directory. These examples demonstrate how to configure the \RateLimitedLayer\ with \tracing\_subscriber\ to rate-limit log messages, showing both basic usage and more complex scenarios where rate limiting is applied based on component and line number attributes.

lib/tracing-limit/examples · high confidence

Aggregate transform adds event-time aggregation support

The aggregate transform now supports event-time aggregation, allowing metrics to be grouped into time-based buckets based on their timestamps rather than the time they are processed. This new capability includes configuration options for handling late-arriving events (via \allowed\_lateness\_ms\), managing missing timestamps (either dropping them or using system time), and guarding against clock skew (via \max\_future\_ms\). This complements the existing system-time aggregation modes (Auto, Sum, Latest, Count, Diff, Max, Min, Mean, Stdev) by providing a way to aggregate metrics according to their intrinsic event time.

src/transforms · high confidence

Automated OpenTelemetry Protocol code generation

The library now automatically generates Rust bindings for OpenTelemetry Protocol (OTLP) definitions during the build process. A new build script scans the local proto files and uses tonic to compile them, producing a static descriptor that allows the application to serialize and deserialize OTLP data without requiring external protocol buffer tools at runtime.

lib/opentelemetry-proto · high confidence

Axiom sink now supports regional edge endpoints

The Axiom sink configuration now includes a \region\ field that allows users to specify a regional edge domain (e.g., \mumbai.axiom.co\) for data ingestion. This feature enables sending data to specific geographic endpoints instead of relying solely on the default global API URL or a custom \url\ setting. The \region\ and \url\ options are mutually exclusive, and the sink validates this constraint at configuration time to prevent invalid setups.

src/sinks/axiom · high confidence

Azure Blob sink now supports expanded authentication methods and connection strings

The Azure Blob sink configuration has been updated to support a wider range of Azure authentication types, including Azure CLI, Client Certificate, Client Secret, Managed Identity (with User Assigned ID support), Managed Identity with Client Assertion, and Workload Identity. Additionally, the sink now supports Azure connection strings for Shared Key and Shared Access Signature (SAS) authentication, allowing users to configure the sink using standard Azure storage connection strings instead of explicit credentials.

_src/sinks/azure\common · high confidence

ClickHouse sink now supports ArrowStream format with automatic schema inference

The ClickHouse sink now supports the \ArrowStream\ batch encoding format, allowing users to send data in Apache Arrow IPC streaming format for improved performance and type fidelity. This change introduces a new \arrow\ module that parses ClickHouse table schemas and maps ClickHouse types (including complex types like Arrays, Tuples, Maps, and UUIDs) to Arrow data types. The sink can now automatically fetch and infer the target table schema from ClickHouse at runtime, eliminating the need for manual schema configuration and ensuring that inserted data matches the table structure. Users can enable this by setting \batch\_encoding.codec\ to \arrow\_stream\ in their sink configuration.

src/sinks/clickhouse · high confidence

Datadog APM stats are now computed and sent independently of trace payloads

The Datadog Traces sink now calculates APM statistics from incoming trace events and flushes them to the Datadog Agent on a 10-second interval, separate from the main trace payload stream. This new aggregation logic groups spans by service, name, resource, and status code, computing hit counts, error rates, and duration distributions (using DDsketch summaries) before sending them as compressed MessagePack payloads. This ensures that statistical data is delivered to Datadog even if the primary trace ingestion path is delayed or disabled.

_src/sinks/datadog/traces/apm\stats · high confidence

Documentation of new Vector capabilities and internal processes

This update adds several new Request for Comments (RFC) documents to the \rfcs\ directory, detailing proposed features and architectural changes. These include API extensions for the \lua\ transform, a comprehensive Kubernetes integration strategy, an event-driven observability model, and automatic request limit adjustment. Additionally, supporting assets such as deployment topology diagrams and icons for the Kubernetes integration are included.

rfcs · high confidence

Elasticsearch sink rewritten to new streaming model with OpenSearch Serverless support

The Elasticsearch sink has been completely rewritten to use the new streaming sink model, introducing a modular architecture with separate modules for configuration, encoding, health checking, and retry logic. This update adds support for Amazon OpenSearch Serverless, requiring AWS authentication and disabling unsupported features like API version autodetection. The sink now supports multiple endpoints, configurable API versions (V6, V7, V8, or auto-detection), and external document versioning. Partial bulk failure retries are now configurable, and the sink properly handles data stream mode with automatic timestamp field renaming to comply with the Elastic Common Schema.

src/sinks/elasticsearch · high confidence

Humio sinks now support configurable indexing and acknowledgements

The \humio\_logs\ and \humio\_metrics\ sinks have been updated to support the \index\ and \indexed\_fields\ configuration options, allowing users to specify the target repository and tag events with extra fields. Additionally, both sinks now support the \acknowledgements\ configuration, enabling end-to-end delivery confirmation from Humio.

src/sinks/humio · high confidence

Initial Windows MSI installer package for Vector

This change introduces the first Windows installer package for Vector, generating an MSI file via WiX. It includes the build script (\build.sh\) to compile the installer, Wix XML templates (\vector.wxs.tmpl\) defining the installation structure, and custom UI assets (\background.svg\, \banner.svg\). The installer places Vector in Program Files, creates Start Menu shortcuts for the command prompt environment, documentation, and configuration folder, and requires administrator privileges to run.

distribution/msi · high confidence

Initial integration of configuration schema AST for the Configurable derive macro

The \lib/vector-config-macros/src/ast\ module has been introduced to provide the internal abstract syntax tree representation for the \Configurable\ derive macro. This new code parses Rust structs and enums into a structured format that captures serde attributes, doc comments, and custom configuration metadata. It enables the macro to generate accurate configuration schemas by handling complex scenarios such as virtual newtypes, enum tagging strategies (external, internal, adjacent, untagged), and field-level metadata, forming the foundational layer for the new schema-driven configuration system.

lib/vector-config-macros/src/ast · high confidence

Initial release of Vector with AMQP support and new CLI structure

Vector is now available as a standalone application, introducing a new CLI interface with subcommands for configuration management, validation, and topology visualization. This release adds a new AMQP source and sink, enabling integration with RabbitMQ and other AMQP brokers, and includes a new \convert-config\ command to migrate configuration files between TOML, YAML, and JSON formats. The application startup and topology management have been restructured into a new \Application\ and \RunningTopology\ architecture, and the default configuration path has been set to \/etc/vector/vector.yaml\.

src · high confidence

Introduce AWS Kinesis Data Firehose sink

Users can now publish logs to AWS Kinesis Data Firehose using the new \aws\_kinesis\_firehose\ sink. This component supports configurable batching (capped at 4 MB or 500 events per batch), partition key fields, and standard AWS authentication and region settings. It includes a healthcheck that verifies the delivery stream exists and matches the configured name, and it enforces batch size limits at configuration validation time to prevent exceeding AWS API constraints.

_src/sinks/aws\kinesis/firehose · high confidence

Introduce Adaptive Concurrency control for sinks

Added a new adaptive concurrency utility that automatically adjusts the number of concurrent requests based on observed response times, helping sinks better handle backpressure and latency spikes. This feature introduces a configurable \AdaptiveConcurrencySettings\ structure allowing users to tune parameters such as the initial concurrency limit, decrease ratio, EWMA alpha, RTT deviation scale, and maximum concurrency limit. The implementation includes a Tower layer (\AdaptiveConcurrencyLimitLayer\) and service that wraps existing sink logic, integrating with internal metrics to expose current load, in-flight counts, and observed/averaged round-trip times for observability.

_src/sinks/util/adaptive\concurrency · high confidence

Introduce GreptimeDB sink with configurable compression and default settings

This change adds the GreptimeDB sink module, enabling users to send data to GreptimeDB via gRPC. The implementation supports configurable gRPC compression (None, Gzip, or Zstandard) for metrics and establishes default values for the database name and pipeline name for logs. It also defines default batch settings, limiting batches to a maximum of 20 events with a 1-second timeout.

src/sinks/greptimedb · high confidence

Introduce HTTP-based external resources for component validation

The component validation framework now supports external resources, starting with an HTTP implementation. This allows test cases to simulate real-world interactions by acting as an HTTP server or client that exchanges events with the component under test. The new \HttpResourceConfig\ and \ResourceCodec\ types enable the framework to encode and decode events using the same codecs configured in the component, ensuring that validation tests accurately reflect the data formats and framing used in production.

src/components/validation/resources · high confidence

Introduce Loki log protocol buffer definitions and serialization utilities

This change adds the core Loki log protocol buffer definitions and utility code to the \lib/loki-logproto\ crate. It introduces structs for representing log entries (including timestamps, lines, and structured metadata), streams, and batches, along with conversion logic to serialize these structures into the protobuf format expected by Loki. The implementation includes label encoding that filters out reserved labels and sorts remaining ones, ensuring correct serialization for log ingestion.

lib/loki-logproto/src · high confidence

Introduce VRL enrichment table lookup functions

This change adds the \get\_enrichment\_table\_record\ and \find\_enrichment\_table\_records\ VRL functions, allowing users to search enrichment tables for single or multiple matching rows using exact, wildcard, or date-range conditions. The implementation includes a \TableRegistry\ that manages table loading and indexing, and a \Table\ trait that defines the search interface, enabling data enrichment directly within VRL expressions.

lib/vector-vrl/enrichment · high confidence

Introduce \`Configurable\` trait for programmatic configuration schema generation

The \lib/vector-config\ library now provides the \Configurable\ trait and associated derive macros, enabling Vector to programmatically generate JSON schemas for its configuration types. This change introduces a new internal API that describes the shape, metadata, and validation rules of configuration fields, allowing for the creation of a Rust-agnostic schema definition. It includes implementations for standard types (numbers, strings, booleans, options) and specific types like \http::StatusCode\, laying the groundwork for machine-generated documentation and configuration validation.

lib/vector-config/src · high confidence

Introduce \`vector top\` CLI subcommand with gRPC-based real-time monitoring

A new \vector top\ command is added to the CLI, providing a real-time dashboard for Vector components and metrics. It connects to the Vector API via gRPC (defaulting to the local API endpoint) and supports features such as human-readable metric formatting, component filtering (with glob patterns), sorting, and automatic reconnection on connection drops. The command requires a TTY and displays an error if the API is unreachable or not enabled.

src/top · high confidence

Introduce \`vector top\` interactive dashboard

Adds a new \vector top\ CLI subcommand that provides a real-time, interactive terminal dashboard for monitoring Vector components. The dashboard displays metrics such as events in/out, bytes in/out, errors, and (on Unix) memory usage, and allows users to sort, filter, and scroll through the component list using keyboard shortcuts.

lib/vector-top · high confidence

Introduce \`vector-config-macros\` for declarative configuration schema generation

The \lib/vector-config-macros\ crate is introduced, providing the procedural macros that power Vector's configuration system. The \\#\[configurable\_component\]\ attribute macro automatically derives the \Configurable\ trait and \serde\ serialization/deserialization for configuration structs, while also registering component metadata (such as name, description, and type) for schema generation and documentation. The \Configurable\ derive macro generates JSON schema definitions and metadata for structs and enums, including support for complex types like ambiguous enums and virtual newtypes. The \NamedComponent\ derive macro validates and extracts component names from specific attributes (e.g., \source\_component\, \sink\_component\), ensuring that all components have valid, lowercase ASCII names. This change establishes the foundational macro infrastructure for defining, validating, and documenting Vector's configuration schema.

lib/vector-config-macros/src · high confidence

Introduce \`websocket\_server\` sink with message buffering, ACK support, and subprotocol handling

The \websocket\_server\ sink is now available, allowing you to deliver observability event data to WebSocket clients. This new sink supports configurable message buffering with replay capabilities, enabling clients to request missed events via message IDs. It includes optional ACK support to track received messages, allowing the server to manage replay checkpoints instead of relying on client-side query parameters. Additionally, the sink allows you to specify a WebSocket subprotocol (accepting any or specific ones), configure TLS, set up HTTP server authentication, and add custom extra tags to internal metrics based on headers, query parameters, or client IP addresses.

_src/sinks/websocket\server · high confidence

Introduce dedicated DNS message parser library with EDNS EDE support

This change introduces a new \dnsmsg-parser\ library that provides a dedicated parser for DNS query and update messages, replacing the previous inline or dependency-bound parsing logic. The parser exposes structured types for DNS headers, questions, and record sections, and includes support for EDNS0 Extended DNS Error (EDE) codes, allowing users to interpret extended response codes and associated diagnostic text. It also supports lowercasing hostnames in RData via parser options and handles various DNS record types including HTTPS, SVCB, and DNSSEC records.

lib/dnsmsg-parser/src · high confidence

Introduce dedicated Prometheus text-format parser library

A new \lib/prometheus-parser\ crate has been added to handle the parsing of Prometheus exposition format metrics. This library provides structured types for counters, gauges, histograms, and summaries, along with a line-level parser built on \nom\ that validates metric names, labels, values, and timestamps. It also includes support for protocol-buffer-based metric metadata definitions and defines a \MetadataConflictStrategy\ to control how conflicting metadata is handled during ingestion.

lib/prometheus-parser/src · high confidence

Introduce dedicated Sematext logs and metrics sinks

This change adds two new sink components, \sematext\_logs\ and \sematext\_metrics\, to the Vector product. The \sematext\_logs\ sink publishes log events to Sematext by delegating to the existing Elasticsearch sink infrastructure, automatically mapping the \timestamp\ field to \@timestamp\ and the \host\ field to \os.host\ to ensure compatibility. The \sematext\_metrics\ sink publishes metric events using the InfluxDB line protocol, supporting region-based endpoint selection (US or EU) and configurable default namespaces. Both sinks are fully configurable via YAML/JSON schemas and include healthcheck support.

src/sinks/sematext · high confidence

Introduce dedicated unit test framework for Vector configurations

This change adds a new \unit\_test\ module that provides a structured way to define and run unit tests for Vector configurations. It introduces \unit\_test\ source and sink components that allow test inputs to be injected into the topology and outputs to be captured and validated against conditions. The framework supports specifying test inputs via \insert\_at\ targets, validating outputs with VRL conditions, and checking for expected event counts or absence of outputs. This enables users to write declarative YAML-based unit tests that verify the behavior of their Vector pipelines without needing to start a full Vector instance.

_src/config/unit\test · high confidence

Introduce disk buffer v2 with zero-copy serialization and improved crash recovery

The disk buffer implementation has been upgraded to version 2, introducing a new on-disk format that uses the \rkyv\ library for zero-copy serialization and deserialization of records. This change brings a new \BackedArchive\ wrapper for efficient memory access, a robust \checkpoint\_recovery\ module to reconcile on-disk data files with the durable ledger after crashes, and a new \ledger\ structure for tracking reader/writer progress. The new buffer enforces stricter invariants, such as a 128MB maximum data file size and CRC32C checksums for all records, while providing better error handling for corrupted or partial writes.

_lib/vector-buffers/src/variants/disk\v2 · high confidence

Introduce file, GeoIP, and MMDB enrichment tables

Users can now enrich data using three new table types: \file\ (loading static CSV data with configurable headers and delimiters), \geoip\ (querying MaxMind GeoIP2/GeoLite2 databases for IP geolocation and ISP details), and \mmdb\ (generic MaxMind database lookups). These tables are integrated into the Vector configuration schema and can be used within \remap\ transforms to add contextual information to logs.

_src/enrichment\tables · high confidence

Introduce internal metrics subsystem with expiration, label filtering, and DDSketch support

The \lib/vector-core/src/metrics\ module now provides a complete internal metrics implementation. It introduces a custom \VectorRecorder\ and \Registry\ that support per-metric-set expiration (allowing idle metrics to be dropped based on configurable timeouts and label/name matchers) and a \LabelFilter\ that restricts tracing span fields to specific HTTP/gRPC server labels and globally registered component identifiers. Additionally, it adds a \DDSketch\ implementation for high-precision histogram aggregation and a native \Histogram\ storage backend with fixed power-of-two buckets.

lib/vector-core/src/metrics · high confidence

Introduce internal schema definitions and validation for Vector log namespaces

Added new internal schema infrastructure in \lib/vector-core/src/schema\ to define and validate event structures for Vector's log namespaces. This includes the \Definition\ struct to describe event and metadata types along with semantic meaning pointers, and the \Requirement\ struct to specify expected semantic fields (like \message\, \timestamp\, \service\) and validate incoming events against them. This enables components to declare their input schema requirements and ensures type safety and semantic consistency for events using the Vector namespace, while maintaining backward compatibility with the Legacy namespace.

lib/vector-core/src/schema · high confidence

Introduce new InfluxDB logs and metrics sinks

This change adds two new sinks, \influxdb\_logs\ and \influxdb\_metrics\, to deliver log and metric data to InfluxDB. The \influxdb\_logs\ sink allows users to configure the target measurement, custom tags, and InfluxDB version-specific settings (database/retention policy for v1, org/bucket/token for v2), while also supporting log schema customization (host, message, source type keys). The \influxdb\_metrics\ sink supports sending metrics with a default namespace, custom tags, and distribution quantiles, also with version-specific connection settings. Both sinks share common configuration structures for endpoints, TLS, batching, and request settings, and include healthcheck support using the \/ping\ endpoint.

src/sinks/influxdb · high confidence

Introduce new format-based decoders for Avro, GELF, InfluxDB, JSON, Native, Native JSON, OTLP, Protobuf, Syslog, and VRL

The \lib/codecs/src/decoding/format\ module now provides a collection of new deserializers that convert raw byte frames into structured Vector events. This includes decoders for Avro (with Confluent schema ID stripping), GELF (with strict/relaxed validation and lossy UTF-8 options), InfluxDB Line Protocol (outputting metrics), JSON (handling arrays and lossy UTF-8), Vector's native Protobuf and JSON formats, OTLP (supporting logs, metrics, and traces with prioritized parsing), Protobuf (using descriptor files and JSON name mapping), Syslog, and VRL. Each decoder implements a common \Deserializer\ trait, allowing sources to specify the input format explicitly, and they all support the new \LogNamespace\ configuration to ensure correct field mapping for both legacy and Vector-native event schemas.

lib/codecs/src/decoding/format · high confidence

Introduce shared GCS sink infrastructure with configurable storage options

This change introduces a new \gcs\_common\ module that provides the foundational configuration, service, and sink components for Google Cloud Storage sinks. It adds support for configuring GCS storage classes (Standard, Nearline, Coldline, Archive) and predefined ACLs, as well as object-level settings like \content\_encoding\ and \cache\_control\. The implementation includes a robust retry mechanism that handles unauthorized errors, request timeouts, and server errors, along with a healthcheck that validates bucket accessibility. This shared layer enables consistent behavior and configuration across GCS-based sinks.

_src/sinks/gcs\common · high confidence

Introduce shared S3 sink configuration and execution layer

The \src/sinks/s3\_common\ module now provides a unified configuration and runtime foundation for AWS S3-compatible sinks. It introduces \S3Options\ to expose granular object metadata settings—including canned ACLs, grant permissions, server-side encryption (KMS key IDs), storage classes (Standard, Glacier, Intelligent Tiering, etc.), and content headers—along with a partitioner that safely renders key prefixes and encryption keys using confined templates. The new \S3Service\ wraps the AWS SDK client as a Tower service to handle uploads, while the generic \S3Sink\ orchestrates batched, partitioned event delivery with built-in concurrency limits and error handling.

_src/sinks/s3\common · high confidence

Introduce the VRL Web Playground

Adds a new interactive VRL (Vector Remap Language) playground to the Vector website, allowing users to write, run, and share VRL programs against sample events directly in the browser. The playground features a split-pane interface with Monaco-based editors for the VRL program, input event, and output, along with syntax highlighting, a run button, and a share feature that encodes state in the URL. It includes a timezone selector for time-based functions, local run history, and a light/dark theme toggle, with version information for Vector and VRL displayed in the header.

lib/vector-vrl/web-playground/public · high confidence

Introduces async file watcher with gzip support and byte offset tracking

The file source now uses a new \FileWatcher\ implementation that operates asynchronously and supports transparent decompression of gzipped log files. This change enables the inclusion of the original byte offset for each line in the emitted events, allowing users to track exactly where in the source file each event originated. The watcher also handles concatenated gzip streams and provides accurate metadata tracking, such as unread byte counts, even after files are deleted or rotated.

_lib/file-source/src/file\watcher · high confidence

Introduces component specification validation for telemetry metrics

Adds a new \ComponentSpecValidator\ that ensures components emit the correct base set of telemetry metrics (such as events received, sent, errors, and bytes) according to their type (source, transform, or sink). This validator checks that input/output event counts match test expectations and verifies specific metric values against runner metrics, providing operators with confidence that component telemetry adheres to the standard specification.

_src/components/validation/validators/component\spec · high confidence

Introduces component validation framework with metric type definitions

This change adds the core structure for validating Vector components by introducing the \Validator\ trait and the \ComponentSpecValidator\ implementation in the new \src/components/validation/validators\ module. It also defines the \ComponentMetricType\ enum, which maps internal metric categories (such as events received, bytes sent, and errors) to their corresponding Prometheus-style metric names, enabling the validation runner to verify component behavior against expected telemetry outputs.

src/components/validation/validators · high confidence

Introduces controlled gRPC edges and telemetry collection for component validation

The component validation runner now constructs isolated topologies using controlled gRPC input and output edges, allowing test events to be injected into sources and captured from sinks without external dependencies. It also attaches internal logs and metrics sources to a dedicated Vector sink, enabling the runner to collect and verify telemetry data (such as \component\_errors\_total\) during validation.

src/components/validation/runner · high confidence

Introduces core configuration schema primitives and metadata handling

The \lib/vector-config-common\ library now provides the foundational types and utilities for the configuration schema system. This includes the \CustomAttribute\ enum for managing metadata flags and key-value pairs, a \ComponentType\ enum defining supported component categories (such as sources, sinks, and transforms), and a \generate\_human\_friendly\_string\ function that converts technical identifiers into readable labels (e.g., \aws\_s3\ to \AWS S3\). Additionally, it introduces a \Validation\ enum supporting JSON Schema validation formats (like \date\, \email\, \ipv4\) and constraints (length, range), along with numeric bounds constants to ensure safe serialization.

lib/vector-config-common/src · high confidence

Introduces new Vector Lookup v2 path types for configuration

The \lib/vector-lookup\ crate now exposes a new \lookup\_v2\ module containing \ConfigValuePath\, \ConfigTargetPath\, and optional path variants (\OptionalValuePath\, \OptionalTargetPath\). These types wrap VRL's internal path structures to provide safe, validated path handling within Vector configuration files, allowing users to define event paths with proper parsing and serialization support.

lib/vector-lookup · high confidence

Introduces runtime allocation tracking with group-based tracing

The allocator module now provides a \GroupedTraceableAllocator\ that wraps the global allocator to trace memory allocations and deallocations when the \TRACK\_ALLOCATIONS\ runtime flag is enabled. This feature groups events by \AllocationGroupId\, allowing users to distinguish memory usage by context via a \tracing\_subscriber::AllocationLayer\. The implementation includes a thread-local stack to manage nested allocation groups and suspends tracing during internal tracer operations to prevent reentrancy panics.

_src/internal\telemetry/allocations/allocator · high confidence

Introduces runtime transform execution model for user-defined code

Adds a new \RuntimeTransform\ trait and its execution infrastructure in \lib/vector-core/src/transform/runtime\_transform\, enabling user-defined code to be executed within the Vector pipeline. This change introduces a lifecycle model with \init\, \process\, \shutdown\, and timer-based hooks, allowing transforms to manage state and respond to periodic events. The implementation includes a \TaskTransform\ adapter that manages the stream of messages (Init, Process, Shutdown, Timer) and a custom \SelectWeak\ stream combinator to handle concurrent event processing and timer intervals, forming the core runtime for dynamic or script-based transforms.

_lib/vector-core/src/transform/runtime\transform · high confidence

Introduction of a standalone VRL CLI tool

A new command-line interface for the Vector Remap Language (VRL) has been added, allowing users to execute VRL expressions directly from the terminal. The tool leverages the \clap\ library for argument parsing and integrates with the \vector-vrl-functions\ crate to provide access to the full suite of available VRL functions, enabling local testing and debugging of VRL logic outside of the main Vector agent.

lib/vector-vrl/cli · high confidence

Introduction of vector-core library with foundational event definitions

A new \vector-core\ library has been added to the project, providing foundational components for building Vector. This includes the extraction of the \Event\ type and associated protobuf definitions (specifically \proto/event.proto\) into this dedicated module. The library is licensed under the Mozilla Public License 2.0 and includes a build script to compile the protobuf definitions using \prost\_build\, establishing the core data structures required for Vector's operation.

lib/vector-core · high confidence

New AMQP sink for sending events to RabbitMQ

The AMQP sink is now available, allowing you to send events to AMQP 0.9.1 compatible brokers like RabbitMQ. You can configure the target exchange and routing key using templates, and optionally set message properties such as content type, content encoding, expiration, and priority. The sink supports configurable encoding (e.g., JSON), connection string-based authentication, and includes integration tests to verify functionality.

src/sinks/amqp · high confidence

New API configuration options and config format support

The configuration system now includes an \api\ section with \enabled\ and \address\ options, allowing users to control the Vector API's network binding and enable/disable it. Additionally, the configuration loader now supports YAML and JSON formats in addition to the existing TOML, automatically detecting the format from the file extension.

src/config · high confidence

New AWS Kinesis Streams sink with configurable partition keys and partial failure retries

This change introduces the \aws\_kinesis\_streams\ sink, allowing you to publish logs directly to AWS Kinesis Streams. The sink supports specifying a \partition\_key\_field\ to control data distribution across shards and includes built-in retry logic for partial record failures, ensuring that individual record errors do not cause the entire batch to be dropped.

_src/sinks/aws\kinesis/streams · high confidence

New AppSignal sink for logs and metrics

A new AppSignal sink has been added, allowing Vector to deliver log and metric events to the AppSignal Push API. The sink supports both log and metric data types, normalizes metrics (ensuring counters are incremental and gauges are absolute), and allows configuration of the API endpoint, push API key, compression, batching, request retries, and TLS settings. Integration tests verify correct payload structure and authentication headers.

src/sinks/appsignal · high confidence

New Azure Logs Ingestion sink for sending logs to Azure Monitor

Users can now configure the \azure\_logs\_ingestion\ sink to publish log events directly to the Azure Monitor Logs Ingestion API. This new capability allows Vector to forward logs to a specified Data Collection Endpoint, targeting a specific Data Collection Rule (via \dcr\_immutable\_id\) and Stream. The sink supports Azure authentication methods (such as client secret and managed identity), configurable timestamp mapping, batch sizing, and retry strategies, enabling reliable ingestion of telemetry into Azure Log Analytics workspaces.

_src/sinks/azure\_logs\ingestion · high confidence

New Databend sink for delivering log data to Databend databases

A new Databend sink has been added, allowing users to deliver log data to a Databend database. The sink supports JSON and CSV encoding, optional Gzip compression, and configurable handling of missing fields for NDJSON inserts. It uses the \databend-client\ library for communication and includes integration tests to verify event delivery.

src/sinks/databend · high confidence

New Databricks Zerobus sink for streaming observability data

A new \databricks\_zerobus\ sink has been added, enabling you to stream observability data to Databricks Unity Catalog tables via the Zerobus ingestion service. The sink supports OAuth 2.0 client credentials authentication, configurable Arrow IPC compression (LZ4 or Zstandard) for Flight payloads, and a custom \user\_agent\ header. It handles Unity Catalog schema resolution and respects Vector's proxy and retry configurations.

_src/sinks/databricks\zerobus · high confidence

New GCP Cloud Monitoring (Stackdriver Metrics) sink

A new sink has been added to deliver metrics to GCP's Cloud Monitoring system. It accepts Counter and Gauge metrics, normalizes them, and sends them to the Google Cloud Monitoring API endpoint (https://monitoring.googleapis.com/v3/projects/{project\_id}/timeSeries) using the native Hyper 1 HTTP client. The sink supports GCP authentication, configurable batching and request limits, TLS settings, and acknowledgements.

src/sinks/gcp/stackdriver/metrics · high confidence

New GCP Cloud Storage, Pub/Sub, and Stackdriver sinks

This release introduces three new sinks for Google Cloud Platform: \gcp\_cloud\_storage\ for storing observability events in GCS buckets with configurable ACLs, storage classes, and object metadata; \gcp\_pubsub\ for publishing events to GCP Pub/Sub topics with support for acknowledgements and TLS; and \gcp\_stackdriver\ (split into logs and metrics modules) for sending data to Google Cloud Monitoring. These sinks replace older implementations and integrate with the new codec and encoding framework, allowing users to configure encoding, compression, and batch settings for GCP data ingestion.

src/sinks/gcp · high confidence

New Google Chronicle Unstructured Log Sink with Regional Endpoints and Compression

A new \gcp\_chronicle\_unstructured\ sink has been added to store unstructured log events in Google Chronicle. This sink supports sending data to specific Google SecOps regional endpoints (such as EU, US, APAC, and various city-specific regions) or a custom endpoint. It includes built-in support for Gzip compression to reduce bandwidth usage and features a fallback mechanism for the \log\_type\ template, ensuring events are not dropped if the template fails to resolve. The sink automatically partitions events by log type and namespace, handling the batching and request building required for the Chronicle ingestion API.

_src/sinks/gcp\chronicle · high confidence

New GreptimeDB Logs sink for ingesting log data

A new \greptimedb\_logs\ sink has been added, allowing you to ingest log events directly into GreptimeDB via its HTTP API. The sink supports configurable endpoints, table names, database names, and pipeline settings, with optional authentication (username/password), HTTP compression (gzip/zstd), custom query parameters, and custom headers. It automatically partitions log events by database, table, pipeline name, and optional pipeline version, and includes template confinement to prevent injection vulnerabilities in dynamic field values.

src/sinks/greptimedb/logs · high confidence

New GreptimeDB metrics sink

A new sink is available to ingest Vector metrics into GreptimeDB via its gRPC interface. It supports authentication, TLS, and gRPC compression, and includes an optional \new\_naming\ configuration option to use GreptimeDB-prefixed column names (\greptime\_value\, \greptime\_timestamp\) for consistency with other ingestion methods.

src/sinks/greptimedb/metrics · high confidence

New HTTP configuration provider for remote config polling

Users can now configure Vector to fetch its configuration from a remote HTTP endpoint. This new provider polls a specified URL at a configurable interval, supporting custom HTTP headers, TLS settings, and proxy configuration. The fetched configuration can be parsed in YAML, TOML, or JSON formats, and optionally supports environment variable interpolation within the remote config file before loading.

src/providers · high confidence

New Kubernetes E2E test framework library

The \lib/k8s-e2e-tests\ library has been introduced to provide a shared, reusable framework for Kubernetes end-to-end testing. It includes utilities for generating unique test namespaces, configuring Helm chart overrides to support multi-node clusters, and constructing test pods with specific affinity rules to ensure logs are collected correctly. Additionally, it provides a dedicated metrics module that parses Prometheus exposition format responses to validate Vector's internal metrics (such as \vector\_started\ and event counts) and host metrics, ensuring the system is healthy and processing data as expected during tests.

lib/k8s-e2e-tests/src · high confidence

New Kubernetes E2E test framework library

The \lib/k8s-test-framework\ library has been introduced to provide a Rust-based API for end-to-end Kubernetes testing. It wraps \kubectl\ and Helm CLI commands to manage test resources, allowing users to deploy Vector via Helm charts, create and manage namespaces and test pods, and perform operations like log tailing, port forwarding, and rollout status checks. The framework includes a \Framework\ struct for high-level orchestration, an \Interface\ for configuring CLI commands via environment variables, and utilities for waiting on resource conditions and handling temporary files.

lib/k8s-test-framework/src · high confidence

New NamedInternalEvent derive macro for stable event identification

The \vector-common-macros\ library now provides a \\#\[derive(NamedInternalEvent)\]\ macro that automatically implements the \NamedInternalEvent\ trait for structs. This allows internal event types to expose a stable, compile-time string identifier (the struct name) via a \name()\ method, simplifying observability and debugging without requiring manual implementation of the trait for each event type.

lib/vector-common-macros · high confidence

New New Relic sink for Events, Metrics, and Logs

A new \new\_relic\ sink has been added, allowing you to deliver logs, metrics, and events to New Relic via the Events, Metrics, and Logs APIs. The sink supports configurable regions (US, EU), API selection, compression, and event encoding via a Transformer. It handles metric conversion (gauges, counters) and log/event mapping, including support for dotted attribute names and JSON parsing within message fields. The implementation includes health checks, retry logic (currently disabled), and compliance with Vector's sink component specifications.

_src/sinks/new\relic · high confidence

New OpenTelemetry HTTP sink for delivering OTLP data

A new \opentelemetry\ sink has been added, allowing users to deliver OpenTelemetry Protocol (OTLP) data over HTTP. The sink currently supports the HTTP protocol and reuses the existing HTTP sink configuration for connection details, encoding, and batching. It includes built-in validation to warn users if they configure JSON encoding with batch sizes greater than one, as this produces invalid OTLP request bodies that receivers will reject. The sink also integrates with the system's security confinement features to protect URIs and templated headers.

src/sinks/opentelemetry · high confidence

New VRL functions to inspect and aggregate internal Vector metrics

This change introduces three new Vector Remap Language (VRL) functions—\get\_vector\_metric\, \find\_vector\_metrics\, and \aggregate\_vector\_metrics\—allowing users to query and process Vector's internal operational metrics directly within transformation pipelines. \get\_vector\_metric\ returns the first matching metric object by name and optional tags, while \find\_vector\_metrics\ returns an array of all matching metrics. \aggregate\_vector\_metrics\ computes a single value (sum, average, max, or min) for counter and gauge metrics matching a specific name and tag filter. These functions operate on a periodic snapshot of internal metrics, the refresh interval of which is controlled by the \metrics\_storage\_refresh\_period\ global option.

lib/vector-vrl/metrics · high confidence

New Vector installation script with custom prefix support

The distribution/install.sh script is introduced to provide a unified, Rustup-inspired installer for Vector. It defaults to version 0.58.0 and downloads from install.datadoghq.com. Users can now specify a custom installation directory using the --prefix flag (which also disables PATH modification) or disable PATH updates entirely with --no-modify-path. The script supports multiple architectures including x86\_64 and ARM variants for Linux (glibc and musl) and macOS, and handles archive extraction with configurable directory structures.

distribution · high confidence

New buffer topology builder and ordered acknowledgement system

The buffer subsystem now uses a new \TopologyBuilder\ to construct multi-stage buffer pipelines, allowing configuration of overflow behaviors (block, drop newest, overflow) and integrated usage instrumentation. A new \OrderedAcknowledgements\ system tracks record processing via markers, enabling accurate detection of record gaps and handling of undecodable records to ensure correct space reclamation in disk buffers.

lib/vector-buffers/src/topology · high confidence

New component description and schema generation infrastructure

The \lib/vector-config/src/component\ module now provides the foundational types for describing Vector components and generating their configuration schemas. It introduces the \ComponentDescription\ struct, which registers component metadata (name, label, description) and leverages the \inventory\ crate to collect all registered components of a specific type (Source, Sink, Transform, etc.). This enables runtime introspection, such as listing available component types or generating example configurations via the \GenerateConfig\ trait. Additionally, the module includes logic to generate JSON Schema objects for these components, facilitating automated documentation and validation of configuration structures.

lib/vector-config/src/component · high confidence

New component validation framework for testing Vector components

A new validation framework has been introduced in \src/components/validation\ to allow developers to write structured, deterministic tests for Vector sources, transforms, and sinks. This location provides the core infrastructure for defining test cases (including input events and expected outcomes like success, failure, or partial success), configuring external resources (such as HTTP or gRPC endpoints), and synchronizing the lifecycle of validation tasks to ensure inputs are fully processed and outputs are captured before assertions are made. It exposes a public API via \ValidatableComponent\ and \ValidationConfiguration\ to register components and their specific validation requirements, enabling robust, isolated testing of component behavior.

src/components/validation · high confidence

New datadog-proto crate for Datadog trace and metric types

A new Rust crate, lib/datadog-proto, has been introduced to centralize the Protobuf definitions and generated code for Datadog telemetry. This crate vendors the \datadog.trace\ and \datadog.trace.idx\ schemas (pinned to a specific commit from the DataDog agent repository) and the \datadog.agentpayload\ metrics schema. It includes a build script that compiles these \.proto\ files into Rust types and generates a static file descriptor set, providing a single, version-controlled source for the wire formats used in Datadog trace and metric ingestion.

lib/datadog-proto · high confidence

New datadog\_traces sink for publishing trace events to Datadog

A new \datadog\_traces\ sink has been added, enabling Vector to publish trace events to the Datadog Trace intake. The sink supports configurable compression (defaulting to gzip), batching with a 3MB payload limit, and standard request settings. It automatically computes and sends APM stats payloads independently of the main trace payloads to comply with Datadog's backend expectations. The implementation includes retry logic for forbidden requests and timeouts, and provides integration tests to verify compliance with sink standards.

src/sinks/datadog/traces · high confidence

New doc-builder tool for generating VRL function documentation

An unpublished CLI tool has been added to generate Vector-specific VRL function documentation. It allows users to output function details as a JSON array to stdout (with optional minification) or write individual JSON files per function into a specified directory, leveraging the underlying VRL documentation build logic.

lib/vector-vrl/doc-builder · high confidence

New docs-renderer tool for generating component schema documentation

A new \lib/docs-renderer\ binary has been added to programmatically generate documentation for Vector component schemas. It reads a JSON schema file, queries for base and specific schemas for Sources, Transforms, and Sinks, and renders them into a structured format, providing a foundation for automated documentation generation.

lib/docs-renderer · high confidence

New encoding formats and GELF chunking support

This release introduces several new encoding formats for sinks, including Arrow IPC streaming, Apache Avro, CEF, CSV, GELF, JSON, Logfmt, Native JSON, OTLP, and Parquet. The GELF encoder now supports native chunking to handle large payloads over transports with size limits (e.g., UDP), configurable via \max\_chunk\_size\. The Arrow encoder allows configurable null handling for non-nullable fields. The JSON encoder supports pretty-printing and configurable metric tag value encoding (single, full, or auto). These formats are available in the \lib/codecs/src/encoding/format\ module and can be selected in sink configurations to serialize events into the desired byte frame format.

lib/codecs/src/encoding/format · high confidence

New event core module with unified event types and metadata

The \lib/vector-core/src/event\ module has been restructured to provide a unified \Event\ enum encompassing \LogEvent\, \Metric\, and \TraceEvent\ types, along with a comprehensive \EventMetadata\ structure for tracking source IDs, schema definitions, and finalization state. This change introduces \EventArray\ for handling collections of events, \Discriminant\ for stream partitioning, and \EstimatedJsonEncodedSizeOf\ for performance-optimized size calculations. It also adds \arbitrary\_impl\ for property-based testing of event structures and \proto.rs\ for serialization support.

lib/vector-core/src/event · high confidence

New event type filtering conditions and Datadog Search integration

Users can now filter events based on their specific type using the new \is\_log\, \is\_metric\, and \is\_trace\ conditions, allowing pipelines to process logs, metrics, and traces separately. Additionally, a new \datadog\_search\ condition type has been introduced, enabling users to apply Datadog's native log search query syntax directly within Vector conditions for more familiar and powerful filtering capabilities.

src/conditions · high confidence

New fakedata library for generating synthetic log lines

A new \lib/fakedata\ module has been introduced to provide utilities for generating synthetic log data. This library exposes functions to create realistic log entries in various formats, including Apache Common and Error logs, Syslog (RFC 3164 and RFC 5424), and JSON. It utilizes static pools of fake usernames, domain names, HTTP endpoints, and error messages to construct these lines, supporting testing and development environments that require realistic but non-sensitive log data.

lib/fakedata · high confidence

New file sink for writing observability events to disk

A new \file\ sink has been added, allowing users to output observability events directly to local files. The sink supports configurable file paths with template-based naming (e.g., date-based rotation), idle timeouts for automatic file rotation, and optional compression using Gzip or Zstandard. It also includes features for path confinement to prevent directory traversal, configurable encoding and framing, end-to-end acknowledgements, and internal metrics for monitoring file I/O operations.

src/sinks/file · high confidence

New framing decoders for bytes, character, newline, length, and varint-delimited streams

The framing module in the codecs library has been restructured into distinct, configurable decoders for various byte-stream protocols. Users can now choose from a \BytesDecoder\ for pass-through framing, a \CharacterDelimitedDecoder\ (used by \NewlineDelimitedDecoder\) that supports configurable delimiters, maximum frame lengths, and an \OversizedAction\ (drop or truncate) to handle malformed data, a \LengthDelimitedDecoder\ wrapping \tokio-util\'s codec with configurable field length and endianness, and a new \VarintLengthDelimitedDecoder\ for protobuf-compatible variable-length prefixes. These changes provide more granular control over how incoming byte streams are split into frames, with built-in safeguards against memory exhaustion via configurable length limits.

lib/codecs/src/decoding/framing · high confidence

New framing methods: Varint length-delimited, character-delimited, and raw bytes

The framing module now supports three additional encoding strategies alongside the existing newline and length-delimited options. Users can choose \varint\_length\_delimited\ to prefix data with a variable-length integer (compatible with Protobuf), \character\_delimited\ to append a configurable ASCII character delimiter, or \bytes\ to disable framing entirely for formats that handle their own length encoding. These new methods are exposed via the \FramingConfig\ enum and integrated into the \Framer\ builder.

lib/codecs/src/encoding/framing · high confidence

New gRPC Observability API for component metrics and topology

A new gRPC service has been introduced in src/api/grpc to expose observability data, replacing the previous GraphQL-based approach. This service provides streams for real-time component metrics (such as received/sent bytes and events) and topology changes, allowing clients to monitor system performance and structure via gRPC instead of HTTP/GraphQL.

src/api/grpc · high confidence

New gRPC-based Vector observability client library

The \vector-api-client\ library has been split out from the main Vector codebase and rewritten to use gRPC instead of the previous GraphQL API. This new client provides a Rust interface for interacting with Vector's observability endpoints, including standard gRPC health checks, metadata retrieval, component listing, allocation tracing status, and streaming metrics (heartbeats, uptime, and memory allocation). Users can now integrate with Vector's observability data using a dedicated, type-safe gRPC client library.

lib/vector-api-client/src · high confidence

New generic network connector service for TCP, UDP, and Unix sockets

A new \NetworkConnector\ utility has been added to the sink infrastructure, providing a unified, stream-based service for connecting to remote endpoints. This component supports TCP (with optional TLS and keepalive), UDP, and Unix Domain Sockets (both stream and datagram modes). It standardizes connection handling, DNS resolution, and send-buffer configuration across sinks, replacing previous ad-hoc socket implementations with a consistent, configurable interface.

src/sinks/util/service/net · high confidence

New internal allocation tracking and telemetry

Added a new internal telemetry module for tracking memory allocations and deallocations. This feature introduces a configurable reporting interval (defaulting to 5 seconds) and exposes metrics for allocated and deallocated bytes per component, allowing users to monitor memory usage patterns within the system.

_src/internal\telemetry/allocations · high confidence

New internal telemetry module for allocation tracking

A new \internal\_telemetry\ module has been added to the codebase, exposing an \allocations\ submodule on Unix systems. This change lays the groundwork for tracking memory allocations, supporting the broader effort to add allocation tracing and visualization capabilities to the \vector top\ command.

_src/internal\telemetry · high confidence

New log event field iteration utilities

Added a new \log\ module under \lib/vector-core/src/event/util\ that provides utilities for iterating over log event fields. This includes \all\_fields\ and \all\_fields\_unquoted\ for traversing nested structures into dotted paths, \all\_metadata\_fields\ for metadata-prefixed paths, \all\_fields\_skip\_array\_elements\ to treat arrays as single values, and \all\_fields\_non\_object\_root\ to handle non-object root values as a single 'message' field. The module also exposes a \keys\ function to retrieve just the field paths. These utilities support quoting invalid field names and are designed to replace previous ad-hoc iteration logic.

lib/vector-core/src/event/util · high confidence

New memory enrichment table with probabilistic filters and source capabilities

The \memory\ enrichment table is now available, allowing users to store and look up enrichment data in memory with configurable TTLs and scan intervals. It supports two probabilistic filter modes—Bloom and Cuckoo—configured via the \filter\ option, which trade value storage for reduced memory usage and support for key removal (Cuckoo). The table can also operate as a source, exporting its contents periodically or emitting expired items via a dedicated output port, with options to control batch sizes and post-export cleanup. State preservation across configuration reloads is configurable, and comprehensive internal metrics track reads, insertions, failures, and TTL expirations.

_src/enrichment\tables/memory · high confidence

New metrics buffering and normalization infrastructure for sinks

This change introduces a new metrics buffering and normalization subsystem in \src/sinks/util/buffer/metrics\ to improve how metrics are aggregated, normalized, and split before being sent to sinks. The new \MetricsBuffer\ collects incoming metrics into batches, applying metric-specific compression (such as aggregating distribution samples) and finalization logic. It relies on a new \MetricSet\ backed by an LRU cache with configurable capacity policies (max bytes, max events, and TTL-based eviction) to manage state efficiently. A \MetricNormalizer\ trait and implementation allow sinks to transform metrics (e.g., converting absolute to incremental) while tracking state, and a \MetricSplitter\ with an \AggregatedSummarySplitter\ implementation automatically breaks down Prometheus-style aggregated summaries into their constituent count, sum, and quantile metrics for sinks that cannot handle them natively. This infrastructure replaces or supplements previous ad-hoc buffering logic, providing a reusable, configurable foundation for metric sinks.

src/sinks/util/buffer/metrics · high confidence

New parse\_dnstap VRL function for parsing DNSTAP events

A new \parse\_dnstap\ VRL function has been added to the Vector VRL library, enabling users to parse base64-encoded DNSTAP data directly within VRL expressions. This function exposes structured fields from the DNSTAP protocol, including server identity and version, socket family and protocol (such as UDP, TCP, DoT, DoH, and DoQ), HTTP protocol version, and detailed DNS message components like query/response headers, question, answer, authority, and additional sections. It also exposes message sizes and handles parsing errors gracefully by emitting warnings and including raw data when necessary.

lib/vector-vrl/dnstap-parser · high confidence

New rate-limiting layer for internal logs

A new \tracing-limit\ crate has been added to provide a \tracing-subscriber\ layer that rate limits internal log events to prevent log flooding. The layer groups events by their callsite and the \component\_id\ field, applying independent rate limits to each group. By default, the first occurrence of an event is emitted normally, the second triggers a suppression warning, and subsequent occurrences are silenced until the rate limit window (default 10 seconds) expires. Users can disable rate limiting for specific logs or customize the window duration using the \internal\_log\_rate\_limit\ and \internal\_log\_rate\_secs\ fields.

lib/tracing-limit/src · high confidence

New schema query primitives for programmatic configuration introspection

The \lib/vector-config/src/schema/parser\ module now exposes a new set of primitives that allow users to programmatically query Vector's configuration schema. This includes a \SchemaQuerier\ that loads a schema file and a \SchemaQueryBuilder\ that supports filtering schemas by custom attributes (both flags and key/value pairs). Additionally, a \ComponentSchema\ type has been introduced to represent the schema of individual components, enabling access to component-specific metadata like type and name while excluding common configuration fields. This change lays the groundwork for tools and integrations that need to inspect or validate configuration structures dynamically.

lib/vector-config/src/schema/parser · high confidence

New shared HTTP authentication module with Bearer and Custom VRL strategies

A new \src/common/http\ module introduces a shared authentication configuration (\HttpServerAuthConfig\) for HTTP-based sources and sinks. This change adds support for Bearer token authentication and a new Custom authentication strategy that allows users to write VRL code to validate requests. The Custom strategy also enables authenticated events to be enriched via metadata writes, providing a flexible way to handle complex authentication logic and event transformation within the HTTP server components.

src/common/http · high confidence

New shared file-source-common library for Vector

A new \lib/file-source-common\ crate has been introduced to consolidate shared logic for file ingestion. This library provides a portable abstraction for reading files across platforms (including Windows-specific metadata handling), manages checkpoint persistence to ensure data recovery after restarts, and handles file fingerprinting to track file identity. It also includes a robust buffering mechanism for reading delimited data and defines the internal events used for observability within the file source pipeline.

lib/file-source-common · high confidence

New shared library for Vector core utilities

The \lib/vector-common\ crate has been introduced to centralize shared functionality used across Vector's core components and related crates. This new library provides common utilities including event finalization and tracking (\finalization\, \finalizer\), decompression safety limits to prevent DoS attacks (\decompression\), atomic metrics support (\atomic\), JSON size estimation for metrics (\json\_size\), and component configuration helpers like \ComponentKey\ and \Inputs\. It also includes test utilities for internal event tracking (\event\_test\_util\) and standard data structures like \ByteSizeOf\ implementations.

lib/vector-common/src · high confidence

New sink buffer utilities with compression and partitioning support

This change introduces a new set of buffer utilities in \src/sinks/util/buffer\ to support sink batching. It adds a \Compression\ enum that allows sinks to configure Gzip, Zlib, Zstandard, and Snappy compression, along with a \Buffer\ struct that handles the actual encoding. Additionally, it provides a \PartitionBuffer\ for grouping events by a key and a \JsonArrayBuffer\ for batching JSON values. These components are designed to be used by sinks to manage batch sizes and apply compression before sending data.

src/sinks/util/buffer · high confidence

New streaming primitives for batching, concurrent mapping, and service driving

The \lib/vector-stream\ crate now provides a suite of new asynchronous stream utilities. The \Batcher\ and \PartitionedBatcher\ components allow items from a stream to be grouped into batches based on configurable size, item count, and timeout limits, with support for custom size calculators and reducers. A new \ConcurrentMap\ stream combinator enables mapping items through asynchronous functions with a configurable limit on in-flight tasks, while the \Driver\ component orchestrates the interaction between an input stream and a \tower::Service\, handling batching, readiness polling, and event finalization. Additional utilities include \FuturesUnorderedCount\ for tracking completion counts of unordered futures and \map\_with\_expiration\ for stateful stream mapping with periodic expiration and final flush capabilities.

lib/vector-stream · high confidence

New test utilities for network address allocation and component testing

The \src/test\_util\ module now includes a suite of test utilities designed to improve test reliability and coverage. A new \addr\ module provides thread-safe port allocation with a guard pattern to prevent race conditions during port reuse, including Windows-specific handling for OS-excluded ports. Additionally, a \components\ module introduces a framework for validating component compliance with internal event and metric specifications, defining standard tag sets and metric requirements for sources, sinks, and transforms. The module also exposes mock sources, sinks, and transforms (such as \test\_basic\, \test\_error\, and \test\_backpressure\) to facilitate isolated component testing, alongside HTTP server and proxy helpers for integration tests.

_src/test\util · high confidence

New time-weighted buffer utilization metrics

The stats module now includes a TimeEwmaGauge that exposes buffer utilization metrics using a time-weighted exponential moving average. This allows users to monitor buffer usage with a configurable half-life, providing a smoother view of utilization that accounts for the duration values remain constant between observations.

lib/vector-common/src/stats · high confidence

New unified encoding configuration and encoder architecture

The encoding subsystem has been restructured into a new \lib/codecs/src/encoding\ module, introducing \EncodingConfig\ and \EncodingConfigWithFraming\ to centralize how events are serialized and framed. This change adds a \Transformer\ component that allows users to filter event fields via \only\_fields\ and \except\_fields\ and standardize timestamp formats before serialization. The system now supports a wide range of serializers including JSON, Avro, Protobuf, GELF, CEF, CSV, Logfmt, Text, and Syslog, with automatic framing selection based on the serializer and sink type (e.g., newline-delimited for JSON, length-delimited for Protobuf). Additionally, batch encoding capabilities are introduced for Arrow IPC and Parquet formats, enabling efficient columnar data processing for compatible sinks.

lib/codecs/src/encoding · high confidence

New utility scripts for Vector build, release, and CI workflows

The \scripts\ directory now contains a comprehensive set of new utility scripts to support Vector's build, packaging, and release processes. This includes \build-docker.sh\ for building and pushing multi-arch Docker images (Alpine, Debian, distroless), \package-deb.sh\, \package-rpm.sh\, \package-msi.sh\, and \package-archive.sh\ for creating distribution packages, and \release-s3.sh\ for uploading artifacts to S3 with verification. CI support is enhanced with \ci-free-disk-space.sh\ and \ci-setup-minikube.sh\, while documentation is managed via \check-docs.sh\ and \cue.sh\. Additional scripts handle Kubernetes E2E testing (\test-e2e-kubernetes.sh\, \deploy-chart-test.sh\), manifest generation (\generate-manifests.sh\), checksum creation (\checksum.sh\), and test result uploads (\upload-test-results.sh\).

scripts · high confidence

Pulsar sink refactored to support TLS, OAuth2, and custom retry options

The Pulsar sink has been rewritten to use the modern StreamSink architecture, introducing support for TLS encryption, OAuth2 authentication, and customizable connection retry options. Users can now configure TLS certificates via the new \tls\ field, authenticate using OAuth2 with \oauth2\ configuration blocks, and tune retry behavior with \connection\_retry\_options\. The sink also supports LZ4, Zlib, Zstandard, and Snappy compression, and allows setting partition keys and message properties from event fields.

src/sinks/pulsar · high confidence

Redis sink now supports Lists, Sorted Sets, and Channels, including Sentinel failover

The Redis sink has been rewritten to support multiple Redis data types: Lists (with configurable LPUSH or RPUSH methods), Sorted Sets (with ZADD and configurable scores), and Channels (pub/sub). It also adds Redis Sentinel support, allowing the sink to automatically discover and failover to a new primary node if the current one becomes unavailable. The sink now uses a StreamSink architecture and supports configurable encoding types for input data.

src/sinks/redis · high confidence

Rewrite component documentation and example generation in Rust

The \vdev\ tool now includes native Rust implementations for generating component documentation and configuration examples, replacing the previous Ruby-based scripts. This adds the \vdev build component-docs\ command to parse JSON configuration schemas and generate CUE files, and the \vdev build component-examples\ command to produce YAML configuration examples from the documentation model. This change consolidates the documentation build pipeline into the \vdev\ CLI, improving consistency and reducing external tooling dependencies for developers.

vdev · high confidence

Source sender now supports configurable chunk sizes and event post-processing

The source sender component in vector-core now allows users to configure the batch size (chunk size) for source events and attach a post-processing step that mutates events just before they are placed on the output channel. This post-processing runs globally across all outputs (default and named ports) and is applied before schema metadata is attached, enabling transformations like enrichment or field modification at the source boundary. Additionally, the sender now exposes configurable timeouts and EWMA half-life settings for buffer utilization metrics, improving observability and control over source event flow.

_lib/vector-core/src/source\sender · high confidence

Splunk HEC sink now supports indexer acknowledgements for reliable delivery

The Splunk HEC sink now integrates with Splunk HEC indexer acknowledgements, providing end-to-end delivery confirmation for sent events. Users can enable this feature via the \indexer\_acknowledgements\_enabled\ configuration option, which controls whether the sink queries the Splunk indexer for final delivery status. The implementation includes configurable parameters for \query\_interval\ (time between status checks), \retry\_limit\ (maximum query attempts per event), and \max\_pending\_acks\ (backpressure threshold). When enabled, the sink tracks acknowledgement IDs returned by Splunk, polls for their status, and finalizes events as delivered, rejected, or errored based on the indexer's response. If the acknowledgement service is unavailable or returns errors, the sink falls back to acknowledging events based on the initial HTTP 200 OK response, ensuring continuity even when Splunk's acknowledgement infrastructure is temporarily unreachable.

_src/sinks/splunk\hec/common · high confidence

Support loading secrets from AWS Secrets Manager, files, and directories

Users can now configure Vector to retrieve secrets from external backends instead of storing them in plaintext. The new \aws\_secrets\_manager\ backend (behind the \secrets-aws\_secrets\_manager\ feature) fetches secrets from AWS Secrets Manager using standard AWS authentication and region configuration. Additionally, the \file\ backend reads a single JSON file containing key-value pairs, and the \directory\ backend reads individual files from a specified path (with an optional \remove\_trailing\_whitespace\ option). The existing \exec\ backend has been updated to support protocol version 1.1, allowing users to pass a \backend\_type\ and \backend\_config\ to the external command. These backends are configured under the \secrets\ section of the Vector configuration.

src/secrets · high confidence

TLS library moved to vector-core with runtime-swappable acceptors

The TLS implementation has been relocated to \lib/vector-core/src/tls\, introducing a new \TlsAcceptorReloader\ that allows TLS certificates to be swapped at runtime without restarting the service. This change adds \bind\_reloadable\ to \MaybeTlsSettings\ to support dynamic certificate rotation for incoming connections, while also including new \MaybeTlsListener\ and \MaybeTls\ types to handle both raw and TLS-wrapped streams uniformly.

lib/vector-core/src/tls · high confidence

Updated Loki Protocol Buffer definitions to support structured metadata

The Protocol Buffer definitions in this library have been updated to include support for structured metadata in log entries. Specifically, the \EntryAdapter\ message in \push.proto\ now contains a \structuredMetadata\ field, allowing clients to send and receive structured metadata alongside log lines. Additionally, the \stats.proto\ definitions have been extended to track statistics related to structured metadata processing, such as \totalStructuredMetadataBytesProcessed\ and \headChunkStructuredMetadataBytes\, providing more detailed insights into query performance and data handling.

lib/loki-logproto/proto · high confidence

VRL Playground now exposes version metadata and supports timezone-aware execution

The VRL Playground library now exposes the Vector and VRL versions (along with their documentation links) to the user interface via new \vector\_version\, \vector\_link\, \vrl\_version\, and \vrl\_link\ functions, ensuring users can see exactly which engine versions are being used. Additionally, the playground's execution engine now accepts a timezone parameter, allowing VRL programs to run with specific timezone contexts rather than relying solely on the default, and includes performance timing data in the compilation results to show execution duration.

lib/vector-vrl/web-playground/src · high confidence

VRL Web Playground now displays specific Vector and VRL version information

The VRL Web Playground UI now exposes the exact Git commit SHA of the Vector repository and the specific version (or Git reference) of the VRL library it is running. This is achieved via a new \build.rs\ script that reads the project's \Cargo.lock\ and Git history at compile time to generate Rust constants, ensuring users can verify which exact versions of the underlying components are active in the playground.

lib/vector-vrl/web-playground · high confidence

Vector Core library initialization with foundational components

The \lib/vector-core\ library has been introduced as a new foundational crate, extracted from the top-level project to provide core building blocks for Vector. This location contributes the \Fanout\ mechanism for distributing events to multiple sinks, the \VectorSink\ abstraction for handling event streams, and the \LatencyRecorder\ for tracking component latency metrics. It also includes the \IpAllowlistConfig\ for network access control, the \Partitioner\ trait for data partitioning, and the \compile\_vrl\ function which enforces read-only access to the 'vector' metadata path during VRL compilation. Additionally, it provides utilities for TCP keepalive configuration, span field registration for observability, and time management via the \KeyedTimer\ trait.

lib/vector-core/src · high confidence

Architecture

New sink utility library for authentication, batching, and encoding

The \src/sinks/util\ module has been restructured into a dedicated library crate (\vector-sinks-util\) that provides shared infrastructure for all sinks. This includes a new \Auth\ enum supporting Basic and AWS authentication, a generic \BatchConfig\ system with predefined settings for real-time and bulk batching, and a \Compressor\ supporting Gzip, Zlib, Zstd, and Snappy. The module also introduces a \SinkBuilder\ trait with stream-based helpers for partitioned batching, concurrent request building, and incremental encoding, alongside utilities for datagram sending and event encoding that replace previous ad-hoc implementations.

src/sinks/util · high confidence

New vector-lib crate consolidates Vector's internal dependencies

A new \vector-lib\ crate has been introduced to serve as a unified facade for Vector's internal libraries, re-exporting key modules such as \vector-core\, \vector-config\, \vector-buffers\, \vector-stream\, \vrl\, and \vector-tap\. This change simplifies dependency management for downstream crates by providing a single entry point for common types, macros (like \compile\_vrl\), and configuration helpers. The library also exposes specific sub-modules for OpenTelemetry and Prometheus parsers when their respective features are enabled, and includes a test to verify that the \register\_extra\_span\_field\ macro correctly registers metric labels without requiring direct inventory dependencies.

lib/vector-lib · high confidence

Splunk HEC sink module structure reorganization

The Splunk HEC sink module has been restructured to separate concerns into distinct sub-modules for common logic, logs, and metrics. This change introduces a new \mod.rs\ file that exposes \common\, \logs\, and \metrics\ modules, facilitating a cleaner architecture for handling different data types within the Splunk HEC integration.

_src/sinks/splunk\hec · high confidence

Topology subsystem refactored into modular builder and controller components

The topology subsystem has been restructured to improve maintainability and separation of concerns. The \src/topology\ module now exposes a dedicated \builder\ module that handles the construction of topology pieces (sources, transforms, sinks) and a \controller\ module that manages the running topology lifecycle, including configuration reloads and graceful shutdowns. This change introduces a \SharedTopologyController\ to coordinate these operations and a \ReadyArrays\ stream combinator to optimize event batching under load. The refactoring also integrates schema definition resolution into the topology building process via a new \schema\ module, ensuring that component inputs and outputs are validated against their schema definitions during the build phase.

src/topology · high confidence

Vector Tap logic extracted into a reusable library

The internal implementation of the Vector Tap feature has been refactored into a standalone library (\lib/vector-tap\). This change extracts the core tap controller, topology monitoring, and notification logic into reusable modules, allowing the tap functionality to be shared across different parts of the application (such as the CLI and the new gRPC API) without code duplication. Users benefit from a more stable and modular tap experience, as the underlying mechanics are now decoupled from specific transport layers.

lib/vector-tap · high confidence

Behavioural changes

AWS CloudWatch Logs sink re-architected with new SDK, retention, and KMS support

The \aws\_cloudwatch\_logs\ sink has been rewritten to use the modern AWS SDK for Rust, replacing the legacy Rusoto client. This update introduces several new configuration options: you can now set a retention policy for newly created log groups, specify a KMS key for encryption, and apply tags to log groups and streams. The sink also supports dynamic log group and stream creation, enforces AWS batch size limits, and includes improved retry logic for throttling errors. Additionally, the healthcheck has been updated to handle dynamic group names and missing groups gracefully.

_src/sinks/aws\_cloudwatch\logs · high confidence

AWS S3 sink restructured with codec support and virtual addressing

The AWS S3 sink has been restructured to support modern codec-based encoding, including a new Parquet batch encoding option (when the codecs-parquet feature is enabled) and configurable text/JSON serialization. Users can now control the S3 addressing style via the new \force\_path\_style\ configuration option to switch between virtual-hosted-style and path-style URLs. The sink also introduces configurable retry strategies for failed requests and allows setting a custom timezone for object key timestamps. These changes are part of a broader migration to the AWS SDK and unified sink configuration model.

_src/sinks/aws\s3 · high confidence

Add Lua bindings for Vector events using mlua

The Lua transform now uses the mlua library to expose Vector events to Lua scripts. This change introduces serialization and deserialization logic for Log and Metric events, allowing scripts to read and write event data via Lua tables. It also adds support for metric-specific fields such as tags, kinds, and values, while explicitly blocking Trace events which are not supported in this context.

lib/vector-core/src/event/lua · high confidence

Add duration, metadata, and reconnection options to the Vector tap subcommand

The \vector tap\ CLI command now supports sampling events for a limited time via the new \--duration\ flag, which causes the command to exit automatically after the specified period. Users can also include component metadata in the output using the \--meta\ flag and control automatic reconnection behavior with the \--no-reconnect\ flag to stop retrying on connection drops. These options are implemented in the \src/tap\ module, which defines the CLI arguments and manages the connection loop for the tap runner.

src/tap · high confidence

Added standard license text files to the repository

The \licenses/\ directory now includes the full text for thirteen common open-source licenses (0BSD, Apache-2.0, BSD-2-Clause, BSD-3-Clause, BSL-1.0, BlueOak-1.0.0, CC0-1.0, CDLA-Permissive-2.0, ISC, MIT, MIT-0, MPL-2.0, OpenSSL, Unicode-3.0, Unicode-DFS-2016, and Zlib). This change provides a centralized location for license texts, supporting compliance and distribution requirements for third-party dependencies.

licenses · high confidence

Azure Blob sink rewritten with new SDK, append blob support, and encoder integration

The Azure Blob sink has been completely rewritten to use the latest Azure SDK, introducing support for append blobs (via the \blob\_type\ option) for continuous log streaming, integration with the \encoding::Encoder\ framework for flexible data serialization, and expanded authentication options including OAuth and default credentials. The configuration now validates at compile time, supports blob index tags and metadata, and enforces stricter security by confining template rendering to prevent injection.

_src/sinks/azure\blob · high confidence

Blackhole sink rewritten with configurable reporting and rate limiting

The blackhole sink has been rewritten to use a new configuration structure that allows users to control periodic activity reporting via the \print\_interval\_secs\ setting (disabled by default) and to enforce a consumption limit using the optional \rate\ field. The implementation now tracks and emits \EventsSent\ and \BytesSent\ metrics, supports end-to-end acknowledgements, and validates that the rate is greater than zero if specified.

src/sinks/blackhole · high confidence

Console sink rewritten with configurable encoding and acknowledgement support

The console sink has been completely rewritten to support configurable output encoding (via the new \encoding\ configuration field) and optional acknowledgements. Users can now specify the output target (stdout or stderr) and define how events are encoded (e.g., JSON, newline-delimited) before they are written to the console. This change also introduces end-to-end acknowledgement support, allowing users to track whether events were successfully processed and written, and integrates the sink with the new observability events (\BytesSent\, \EventsSent\) for better monitoring.

src/sinks/console · high confidence

Datadog Events sink rewritten to new sink model with field filtering

The \datadog\_events\ sink has been rewritten to the new sink architecture, introducing stricter validation and automatic field mapping. The sink now requires the \title\ field and automatically maps the event \message\, \host\, \timestamp\, and \source\_type\_name\ to the Datadog API fields \text\, \host\, \date\_happened\, and \source\_type\_name\ respectively if they are missing. Additionally, the sink now strictly filters outgoing payloads to only include fields accepted by the Datadog Events API (such as \title\, \text\, \host\, \tags\, etc.), dropping any other fields to prevent API rejection.

src/sinks/datadog/events · high confidence

Datadog Logs sink rewritten to use the v2 API with zstd compression and configurable payload limits

The Datadog Logs sink has been completely rewritten to target the Datadog Logs v2 API endpoint (/api/v2/logs) instead of the legacy v1 endpoint. This change introduces zstd compression by default to reduce bandwidth usage, adds a configurable \max\_payload\_bytes\ setting (defaulting to 5 MB) to control batch sizes, and enforces stricter payload size limits to prevent API rejections. The sink now normalizes events to conform to the Datadog Agent standard when the \conforms\_as\_agent\ option is enabled, ensuring consistent field mapping and timestamp formatting. Additionally, the rewrite includes improved error handling, retry logic for transient failures, and better telemetry reporting for bytes sent and events delivered.

src/sinks/datadog/logs · high confidence

Datadog Metrics sink rewritten with V3 API support and new architecture

The \datadog\_metrics\ sink has been completely rewritten to support the new V3 columnar protobuf API endpoint (\/api/intake/metrics/v3/series\), which is now the default series API version. This change introduces a new encoder architecture that uses V3 for series metrics while retaining the V1/V2 encoder for sketches, and includes a new normalizer to handle metric types like distributions and aggregated histograms. The sink now validates configuration at compile time, uses zstd compression for V2/V3 endpoints, and enforces stricter payload size limits (5 MiB for V2/V3 vs 60 MiB for V1).

src/sinks/datadog/metrics · high confidence

Default sink concurrency changes to adaptive

The default concurrency mode for sinks is now adaptive, meaning outbound requests are managed by Vector's Adaptive Request Concurrency (ARC) feature by default rather than using a fixed limit. This change is implemented via a new \Concurrency\ enum in the sink utility service layer, which supports \adaptive\, \none\ (fixed to 1), or a specific integer limit, and includes updated serialization and configuration schema generation to reflect this new default behavior.

src/sinks/util/service · high confidence

Delayed Kubernetes resource deletion for event enrichment

The Kubernetes module now includes a \custom\_reflector\ that delays the application of \Delete\ events for Kubernetes resources by a configurable duration. This allows the system to continue enriching log events with metadata (such as pod labels and annotations) even after a resource has been removed from the cluster, preventing data loss during rapid pod churn. The implementation uses a \MetaCache\ to track active metadata and a \DelayQueue\ to defer deletion processing, ensuring that downstream consumers have sufficient time to process the final state of the resource.

src/kubernetes · high confidence

GCP Stackdriver Logs sink refactored with template confinement and retry strategy

The \gcp\_stackdriver\_logs\ sink has been refactored to use the new \vector\_lib\ sink architecture, introducing stricter security by confining routing-field templates to prevent injection attacks. The configuration now supports a \RetryStrategy\ for handling transient failures and allows additional top-level labels to be configured. Users will see improved reliability and security posture when delivering logs to GCP Cloud Operations.

src/sinks/gcp/stackdriver/logs · high confidence

HTTP client implementation migrated to Hyper 1

The HTTP client in src/http has been updated to use Hyper 1, introducing a new client implementation (client\_v1.rs) that replaces the previous version. This change includes updated transport logic, TLS handling, and proxy support, along with new contract tests (transport\_tests.rs) to verify behavior across both legacy and new client versions.

src/http · high confidence

HTTP sink refactored to support templateable URIs and headers with runtime validation

The HTTP sink has been rewritten to support dynamic, template-based URIs and request headers, allowing users to inject event data into request targets and headers. This change introduces runtime validation for these templates to prevent injection attacks and ensure correct formatting. Additionally, the sink now supports configurable HTTP methods (GET, POST, PUT, DELETE, etc.), payload prefixes and suffixes for custom JSON wrapping, and improved batching based on the configured encoder's output size.

src/sinks/http · high confidence

Honeycomb sink now supports configurable endpoints, compression, and retry strategies

The Honeycomb sink has been refactored to allow users to configure the API endpoint (defaulting to https://api.honeycomb.io), enabling support for regional endpoints like api.eu1.honeycomb.io. It now supports request compression (defaulting to zstd) to reduce bandwidth usage and includes a configurable retry strategy to handle transient network failures more gracefully. The sink also validates that the endpoint is a valid absolute HTTP(S) URL at configuration compile time.

src/sinks/honeycomb · high confidence

Import of third-party Google API and Protobuf definitions

The \proto/third-party/google\ directory now contains vendored copies of Google's standard API and Protobuf definitions, including \annotations.proto\, \client.proto\, \field\_behavior.proto\, \http.proto\, \resource.proto\, \any.proto\, \descriptor.proto\, \empty.proto\, and \timestamp.proto\, along with their respective Apache 2.0 and BSD licenses. This change provides the local schema definitions required for gRPC transcoding, resource descriptors, and well-known types, supporting the broader migration to gRPC by ensuring the necessary protobuf contracts are available within the repository.

proto · high confidence

Improved config schema output with precise validation and human-friendly names

The generated configuration schema now enforces strict validation by disallowing unknown properties (equivalent to \deny\_unknown\_fields\), ensuring that any unrecognized fields in a configuration file trigger a validation error. To support this, the schema structure has been optimized by flattening nested references where possible and inlining single-use references to reduce complexity. Additionally, the schema now includes human-friendly display names for enum variants and properties, making documentation and error messages more readable for users.

lib/vector-config/src/schema/visitors · high confidence

Improved decoder error resilience and GELF codec support

The codecs library now includes a \DecoderFramedRead\ wrapper that prevents the decoding stream from terminating on recoverable errors, allowing sources to continue processing valid data after malformed input. This change is accompanied by the addition of GELF codec support (including field definitions and validation) and new internal events to better track and report decoder framing, deserialization, and serialization errors.

lib/codecs/src · high confidence

Introduce asynchronous file source with configurable fingerprinting and checkpointing

The file source implementation has been refactored into an asynchronous architecture using Tokio, introducing a new \FileServer\ that cooperatively schedules reads across files. This change adds support for configurable fingerprinting strategies, checkpoint persistence to a single JSON file, and options to remove files after reaching EOF or ignoring small files. Users benefit from improved performance through batched line reading, backoff for inactive files, and more robust handling of file rotations and shutdown signals.

lib/file-source/src · high confidence

Introduce base Cross Dockerfile and Ubuntu bootstrap script

The CI infrastructure for cross-compilation now uses a dedicated base Dockerfile and bootstrap script. The new Dockerfile initializes the cross-rs environment, installs build dependencies via the bootstrap script, and applies configuration changes to allow CMake to locate pre-built dependencies outside the sysroot. Additionally, it includes logic to relocate libstdc++ for specific musl-based ARM targets to ensure compatibility during the build process.

scripts/cross · high confidence

Introduce multi-output transform architecture with runtime schema propagation

Transforms now support multiple named outputs, allowing events to be routed to different downstream components based on logic within the transform. This change introduces a new \TransformOutputs\ and \TransformOutputsBuf\ abstraction to manage event buffering and sending to these distinct ports. Additionally, when events are sent to a specific output, the system automatically updates the event's runtime schema definition to reflect the target output's schema, ensuring downstream components receive events with the correct type information.

lib/vector-core/src/transform · high confidence

Introduce registered internal event system for component metrics

The internal observability layer in \lib/vector-common\ has been refactored to use a new 'registered event' architecture. This change introduces typed event structs (such as \BytesReceived\, \BytesSent\, \EventsReceived\, \EventsSent\, and \ComponentEventsDropped\) that wrap the underlying metrics library to ensure metrics with dynamic tags are properly registered and retained, preventing counter resets. It also adds a \RegisteredEventCache\ to manage these dynamic tag registrations and a \NamedInternalEvent\ trait for consistent event identification. For users, this provides a more robust and consistent foundation for internal component metrics, ensuring that per-component counters (like \component\_received\_events\_total\) remain stable and accurate even when emitting events with varying tags.

_lib/vector-common/src/internal\event · high confidence

Introduce shared Datadog sink configuration and test utilities

The Datadog sinks now share a common configuration structure (\LocalDatadogCommonConfig\ and \DatadogCommonConfig\) that centralizes settings for the API endpoint, Datadog site, default API key, TLS, and acknowledgements. This unification ensures consistent configuration handling across logs, metrics, traces, and events sinks, while also introducing a dedicated healthcheck mechanism that validates connectivity to the Datadog API. Additionally, a new \test\_utils\ module has been added to provide standardized test server helpers for validating API status codes across the different sink implementations.

src/sinks/datadog · high confidence

Introduce shared infrastructure for retry logic, component configuration, and AWS client building

This change establishes a new \src/common\ module that consolidates shared logic across Vector components. It introduces an \ExponentialBackoff\ strategy to standardize retry delays, which is now used by WebSocket connectors to handle connection failures with configurable timeouts and infinite retries. It also adds common configuration structs for Datadog (handling API endpoints, site prefixes, and metric metadata), MQTT (broker settings, TLS, and packet size limits), and AWS services (S3 and SQS client builders using the AWS SDK). Additionally, it provides utility functions for tag expansion in the \log\_to\_metric\ transform and Loki sink, and standardizes error handling and connection logic for WebSocket sources and sinks.

src/common · high confidence

Introduces new topology channel implementation with overflow support and detailed metrics

The buffer topology channels have been replaced with a new implementation that supports overflow configurations, allowing events to spill from a base buffer to an overflow buffer when capacity is reached. The new channel provides granular observability through standardized buffer utilization metrics, including time-weighted exponential moving average (EWMA) means for both source and transform buffers, as well as specific metrics for buffer send duration. It also introduces robust handling for disk v2 buffer errors, ensuring that recoverable errors do not cause panics and that unencodable items are correctly tracked and dropped rather than crashing the pipeline.

lib/vector-buffers/src/topology/channel · high confidence

Kafka sink rewritten with new-style sink architecture and enhanced configuration

The Kafka sink has been completely rewritten to use the new-style sink framework, introducing a new configuration structure (\KafkaSinkConfig\) that supports templated topics, configurable healthcheck topics, and explicit rate limiting options (\rate\_limit\_num\, \rate\_limit\_duration\_secs\). The sink now supports extracting message keys and headers from log, trace, and metric events, and includes improved error handling for producer queue full and policy violation errors with automatic retry logic. The implementation also integrates with the new encoding/transformer pipeline and provides detailed metrics for events sent and bytes sent.

src/sinks/kafka · high confidence

Loki log protocol library restructured with new build and licensing

The \lib/loki-logproto\ crate has been reorganized to support third-party protocol definitions. A \build.rs\ script was added to compile specific Protocol Buffers files (\gogo.proto\, \stats.proto\, \logproto.proto\, \push.proto\) using \prost\_build\, indicating a shift in how the protobuf definitions are generated. Additionally, a \LICENSE\ file adopting the Mozilla Public License 2.0 was introduced for this module, clarifying the licensing terms for the protocol definitions contained within.

lib/loki-logproto · medium confidence

Loki sink rewritten with new streaming architecture and structured metadata support

The Loki sink has been completely rewritten to use a new streaming-based architecture, replacing the previous implementation. This update introduces support for structured metadata, allowing users to attach dynamic key-value pairs to log batches alongside standard labels. The default behavior for out-of-order events has changed to 'Accept', letting Loki handle reordering (requiring Loki 2.4.0+), and Snappy compression is now the default, sending requests as Protocol Buffers. The sink also supports configurable URL paths, dynamic tenant IDs, and a customizable healthcheck URI.

src/sinks/loki · high confidence

Metric data model refactored into Metric, MetricData, and MetricSeries structs

The internal metric representation has been restructured to separate concerns: the \Metric\ struct now acts as a container holding a \MetricSeries\ (name and tags), a \MetricData\ struct (timestamp, interval, kind, and value), and event metadata. This change introduces an \interval\_ms\ field to \MetricTime\ to support rate tracking on counters, adds an \Incremental\ metric kind alongside \Absolute\, and provides methods like \into\_absolute\, \into\_incremental\, and \update\ to manage metric state transitions and aggregation. The \MetricValue\ enum remains the carrier for specific metric types (Counter, Gauge, Histogram, etc.), but is now accessed via the \MetricData\ layer.

lib/vector-core/src/event/metric · high confidence

Migration to gRPC-based observability client with updated build configuration

The Vector API client has switched from a GraphQL-based interface to a gRPC-based one for observability data. This change updates the build process to generate Rust code from \observability.proto\ and \event.proto\ files using \tonic\ and \prost\, replacing the previous \async-graphql\ dependencies. Additionally, a Mozilla Public License 2.0 file has been added to the crate, and pedantic linting has been enabled for the build script.

lib/vector-api-client · high confidence

Modernized CI environment setup with centralized tooling and Windows 2025 support

The CI environment setup has been restructured to improve reliability and consistency across platforms. A new \prepare.sh\ script centralizes the installation of Rust tooling and Cargo utilities (such as cargo-nextest, cargo-deny, and wasm-pack) using pinned versions, replacing scattered installation logic. Support for the Windows 2025 runner image has been added via a dedicated bootstrap script that configures paths for Rust, protoc, and OpenSSL, while removing the dependency on Chocolatey. Additionally, a new \binstall.sh\ script enables the use of \cargo-binstall\ for faster, binary-based installation of tools, and shared scripts for installing protoc and Debian build dependencies ensure consistent environments across Linux, macOS, and Windows.

scripts/environment · high confidence

NATS sink refactored to support JetStream and configurable message headers

The NATS sink has been restructured to use the StreamSink component architecture, enabling the use of NATS JetStream for message persistence. Users can now enable JetStream via the \jetstream\ configuration block and attach custom headers to published messages using the \jetstream.headers\ option, such as setting a \message\_id\ template for deduplication. The sink also supports parsing multiple NATS URLs for connection redundancy and includes improved handling for connection failures, ensuring the service behaves correctly based on the \--require-healthy\ flag when DNS resolution or availability fails.

src/sinks/nats · high confidence

New buffer library with standardized metrics and encoding traits

The \lib/vector-buffers\ crate has been restructured to provide a unified interface for memory and disk-backed buffers. This change introduces a new \BufferUsageHandle\ that tracks event counts and byte sizes via atomic counters, emitting standardized metrics for received, sent, and dropped events. It also defines the \Encodable\ and \FixedEncodable\ traits to support schema evolution and metadata-aware encoding for persistent storage, while exposing configuration options for buffer types (memory vs. disk v2) and full-buffer behaviors (block, drop newest, overflow).

lib/vector-buffers/src · high confidence

New gRPC-based observability API with unified health endpoint

The API module now exposes a new gRPC server for observability data, replacing the previous custom health RPC with the standard gRPC health service (grpc.health.v1.Health). This server also serves an HTTP GET/HEAD /health endpoint that returns 200 {"ok":true} or 503 {"ok":false} based on the server's serving status, ensuring consistency between HTTP and gRPC health probes for Kubernetes and load balancers.

src/api · high confidence

New global configuration options for wildcard matching, metric expiration, and telemetry tags

The global configuration now includes a \wildcard\_matching\ option (defaulting to strict) to control whether wildcard input patterns must match existing inputs, and a \metrics\_storage\_refresh\_period\ to configure the internal metrics cache refresh interval for VRL. Internal metric expiration is now configurable via \expire\_metrics\_secs\ (replacing the deprecated \expire\_metrics\) and supports granular per-metric-set expiration through \expire\_metrics\_per\_metric\_set\. Additionally, the \telemetry\ section allows enabling \emit\_source\ and \emit\_service\ tags on component telemetry events.

lib/vector-core/src/config · high confidence

New hardened systemd service and environment file support

The distribution now includes a new 'hardened-vector.service' unit file that applies strict systemd sandboxing (such as ProtectSystem=strict, PrivateTmp, and syscall filtering) to limit Vector's capabilities, alongside a standard 'vector.service' that ensures the service restarts always. Both units support loading environment variables from a new '/etc/default/vector' file, and the reload mechanism has been updated to validate configuration changes safely.

distribution/systemd · high confidence

New modular configuration loading infrastructure with disabled environment variable interpolation by default

The configuration loading logic in \src/config/loading\ has been restructured into a modular system featuring a new \ConfigBuilderLoader\ and dedicated modules for interpolation, secret resolution, and schema coercion. A key behavioral change is that environment variable interpolation is now disabled by default; users must explicitly enable it via the \set\_env\_var\_interpolation\ API or the \--dangerously-allow-env-var-interpolation\ CLI flag to restore previous behavior. The new loader also introduces a \SecretBackendLoader\ to handle \SECRET\[...\]\ placeholders and a \ValueCoercer\ to prepare values against generated JSON schemas before final deserialization.

src/config/loading · high confidence

New unified AWS authentication and configuration module

The \src/aws\ directory has been restructured into a new, shared AWS module (\auth.rs\, \region.rs\, \timeout.rs\, \mod.rs\) that centralizes AWS client construction, region resolution, and timeout handling. This change introduces a structured \AwsAuthentication\ enum allowing users to explicitly configure authentication via fixed Access Keys, credential Files, or assumed Roles, with support for session tokens, external IDs, and session names. It also adds configurable IMDS client settings (retries, connect/read timeouts) and a unified \AwsTimeout\ struct for connection, operation, and read timeouts, replacing previous ad-hoc or SDK-default configurations.

src/aws · high confidence

Prometheus exporter sink rewritten with new configuration and metric collection logic

The \prometheus\_exporter\ sink has been completely rewritten to use a new modular architecture. The \collector.rs\ module now handles the conversion of internal metrics into Prometheus exposition formats, including specific logic for aggregating distributions into histograms or summaries and handling cumulative bucket counts. The \exporter.rs\ module introduces a new \PrometheusExporterConfig\ struct that exposes configuration options for the listening address, authentication, TLS, histogram buckets, summary quantiles, flush periods for metric expiration, and timestamp suppression. This change replaces the previous implementation with a more robust, configurable, and testable codebase for exposing metrics via HTTP.

src/sinks/prometheus · high confidence

Prometheus parser now supports complex metrics and remote write metadata

The Prometheus parser library has been updated to handle complex metric types (such as histograms and summaries) and metadata via the reintroduction of MetricMetadata support in the remote write source. This change is implemented by adding the upstream Prometheus protobuf definitions (prometheus-remote.proto and prometheus-types.proto) to the library, configuring the build script to generate Rust bindings from these protos, and adding the Mozilla Public License 2.0 file to the project.

lib/prometheus-parser · high confidence

Prometheus remote write sink rewritten with new configuration and behavior

The \prometheus\_remote\_write\ sink has been completely rewritten, introducing a new \RemoteWriteConfig\ structure that adds support for custom HTTP headers, a TTL-based metrics cache (\expire\_metrics\_secs\) to prevent indefinite memory growth, and configurable histogram buckets and summary quantiles. The sink now validates configuration at compile time, enforces confinement for template fields, and partitions outgoing requests by tenant ID. It also supports AWS authentication, basic auth, and sends the \X-Scope-OrgID\ header when a tenant ID is configured.

_src/sinks/prometheus/remote\write · high confidence

Redesigned Protobuf schema for events with enhanced metadata and metric support

The core event protocol definition has been updated to support a more structured and extensible event model. Users will benefit from richer metadata tracking, including specific fields for Datadog origin information, source identifiers, and trace layout markers, alongside the introduction of an internal event ID. Metric handling has been improved with the addition of an interval field, support for sketch data types, and a new enhanced tags system (tags\_v2) replacing the deprecated v1 tags. Additionally, the schema now explicitly supports trace data types and allows log events to have non-object root values, providing greater flexibility in how data is serialized and processed.

lib/vector-core/proto · high confidence

Refactor buffer variants to use new in-memory and disk v2 implementations

The buffer implementation in this library has been restructured to expose only the new \DiskV2Buffer\ and \MemoryBuffer\ variants, removing the legacy \disk\_v1\ (LevelDB-based) implementation. The new \MemoryBuffer\ now supports configuration via \MemoryBufferSize\ (either max events or max size) and integrates with buffer utilization tracking through \BufferUsageHandle\. This change simplifies the buffer API by removing obsolete components like the \Acker\ and aligning with the new \Sink\ trait removal in fanout channels.

lib/vector-buffers/src/variants · high confidence

Refactored AWS Kinesis sinks to extract shared configuration and retry logic

The \aws\_kinesis\ sink module has been restructured to share common configuration and logic between the Firehose and Streams sinks. A new \KinesisSinkBaseConfig\ centralizes settings such as stream name, region, encoding, compression, request limits, TLS, authentication, and acknowledgements, which specific sink implementations now extend. The refactoring introduces a generic \KinesisSink\ and \KinesisService\ that handle batching, request building, and partition key generation, while supporting configurable retry logic for partial failures via a new \RetryLogic\ trait and \RecordResult\ tracking in responses. This change improves maintainability and ensures consistent behavior across both Kinesis Firehose and Kinesis Streams sinks.

_src/sinks/aws\kinesis · high confidence

Refactored schema generation with new helper functions and metadata handling

The schema generation logic in \lib/vector-config/src/schema\ has been restructured to improve maintainability and output precision. A new \helpers.rs\ module exposes internal functions for applying metadata (titles, descriptions, defaults, deprecation flags) and validations to JSON Schema objects, ensuring that per-field metadata can override type-level defaults without mixing. The system now strictly disallows unevaluated properties in the generated schema and flattens schema references to reduce complexity. Additionally, the module re-exports core types from \vector-config-common\ and introduces visitors for generating human-friendly names and handling ambiguous enum schemas via \anyOf\ instead of \oneOf\. These changes result in cleaner, more precise configuration schemas with better support for optional and internally-tagged enum variants.

lib/vector-config/src/schema · medium confidence

Refined Debian packaging user setup and log group access

The Debian package installation scripts have been updated to ensure the Vector user is correctly provisioned and granted necessary permissions for log access. The pre-install script now explicitly creates the system user and group, sets up the data directory, and assigns ownership. The post-install script adds the Vector user to the 'adm' group for reading /var/log, and conditionally adds it to 'systemd-journal' and 'systemd-journal-remote' groups if those groups exist, reloading the systemd daemon as needed to apply group changes.

distribution/debian · high confidence

Regression test suite converted to YAML configuration

The regression test cases in the \regression/cases\ directory have been migrated from TOML to YAML format. This change updates the experiment definitions, lading load-generator configurations, and Vector pipeline configurations for all affected cases (including \datadog\_agent\_remap\_blackhole\, \file\_to\_blackhole\, \fluent\_elasticsearch\, \http\_elasticsearch\, \otlp\_grpc\_to\_blackhole\, and various scale tests) to use the new YAML structure, ensuring consistent configuration parsing and easier maintenance for performance and integration testing.

regression/cases · high confidence

Rename LogDNA sink to Mezmo and expand Socket sink modes

The LogDNA sink has been renamed to Mezmo (src/sinks/mezmo.rs) to reflect the service's rebranding, while the Socket sink (src/sinks/socket.rs) now supports TCP, UDP, Unix Stream, and Unix Datagram modes, allowing users to deliver logs to a wider variety of remote socket endpoints.

src/sinks · high confidence

Replace GraphQL observability API with gRPC

The observability API has been migrated from GraphQL to a gRPC-based implementation. This change introduces new protocol buffer definitions and tonic-based client/server structures for the observability service, replacing the previous async-graphql dependencies. The proto module now conditionally exposes the observability service when the 'api' feature is enabled, alongside existing vector source/sink proto definitions.

src/proto · high confidence

Replaced external schemars dependency with an internal JSON Schema 2019-09 generator

The \lib/vector-config-common/src/schema\ module has been rewritten to include a self-contained implementation of JSON Schema generation, replacing the previous external \schemars\ dependency. This new internal generator targets the JSON Schema 2019-09 specification, providing features such as \unevaluatedProperties\ support, schema reference flattening, and ordered output via \BTreeMap\. It includes a visitor system for post-processing schemas and navigation helpers for inspecting configuration structures, effectively decoupling the schema generation logic from the external crate while maintaining compatibility with the existing configuration validation pipeline.

lib/vector-config-common/src/schema · high confidence

Rewrite of the Splunk HEC Logs sink to the new Vector sink architecture

The \splunk\_hec\_logs\ sink has been rewritten to the new Vector sink style, introducing a modular structure with dedicated configuration, encoder, request builder, and sink driver components. This change integrates the sink with the new \encoding::Encoder\ framework, allowing it to support both JSON and text encoding via the \encoding\ configuration option. It also implements security confinement for template fields (index, sourcetype, source) to prevent path traversal attacks, and adds support for namespaceable \host\_key\ and \timestamp\_key\ fields. The rewrite preserves existing features like indexer acknowledgements, compression, and batching while improving internal observability and code maintainability.

_src/sinks/splunk\hec/logs · high confidence

Rewrite of the Splunk HEC metrics sink to the new sink architecture

The Splunk HEC metrics sink has been rewritten to use the new sink style, introducing a dedicated encoder for serializing metrics and a request builder that handles compression and payload construction. Configuration is now validated at compile time using the \configurable\_component\ macro, and the sink supports templated fields (index, source, sourcetype) with security confinement to prevent template injection. The implementation also integrates indexer acknowledgements, telemetry for data volume, and updated integration tests to verify compliance with the new sink driver.

_src/sinks/splunk\hec/metrics · high confidence

Sources now use the new Log Schema and namespace definitions

Sources now support the new log namespace and schema definitions, allowing for more structured and consistent log data handling. This change introduces the ability to define and use schemas for log events, enhancing the clarity and reliability of log data processing.

src/sources · high confidence

Standardize repository formatting and linting configuration

The repository now enforces consistent code style and documentation formatting through new configuration files. Rust code is formatted using \.rustfmt.toml\ (Rust 2024 edition, 100-character line width), while Markdown files are linted via \.markdownlint.jsonc\ (migrated to markdownlint-cli2) and other text files are formatted with \.prettierrc.json\. Editor integration is standardized via \.editorconfig\, and file handling for Git and Docker is defined in \.gitattributes\ and \.dockerignore\ to exclude build artifacts and generated files.

(repo-wide) · high confidence

Standardized internal event instrumentation across components

The internal event definitions for sources, sinks, and transforms have been rewritten to adhere to the component instrumentation specification. This change introduces a consistent, named-event system that emits structured counters (such as ComponentErrorsTotal, ComponentReceivedEventsTotal) and histograms with standardized labels (error\_type, stage, protocol), replacing ad-hoc logging and metric emission. Users will see more uniform and detailed observability data in their internal metrics, with better error categorization and reduced noise from duplicate or inconsistent event names.

_src/internal\events · high confidence

StatsD sink refactored to a stream-based architecture with improved batching and encoding

The StatsD sink has been rewritten to use a modern stream-based processing pipeline, replacing the previous implementation. This change introduces a new encoder that handles metric serialization (counters, gauges, distributions, sets) and a normalizer that ensures counters and distributions are treated as incremental for accurate aggregation. Batching is now driven by a custom \StatsdBatchSizer\ that estimates byte sizes based on metric names, namespaces, and tags, allowing for more efficient request construction. The sink also supports incremental request building to optimize payload sizes for different socket modes (UDP, TCP, Unix domain sockets), ensuring payloads stay within MTU limits for UDP while allowing larger batches for TCP/Unix sockets.

src/sinks/statsd · high confidence

Stricter URI template confinement validates scheme and authority

URI templates used in HTTP/HTTPS sink fields now enforce stricter confinement checks at build time: the static prefix must include a supported scheme (HTTP or HTTPS) and a static host (authority). Templates that rely solely on strftime specifiers without a static scheme and host, or that leave the host event-controlled, are now rejected with clear errors (e.g., NoStaticUriAuthority, UnsupportedUriScheme) instead of allowing potentially unsafe resolution at runtime. This prevents false URL confinement for templates in non-URL fields and ensures that rendered URIs cannot escape the operator-configured base via scheme or host manipulation.

src/template · high confidence

Unified decoding pipeline with configurable framing, decompression, and log namespace support

The decoding subsystem has been restructured into a cohesive pipeline that applies payload decompression (Gzip, Zlib, or Zstandard with optional dictionaries) before framing and deserialization. Users can now configure how byte streams are split into frames using methods such as newline, character, length, octet counting, chunked GELF, or varint length-delimited framing. The decoder respects the configured log namespace (e.g., OpenTelemetry vs. legacy) when parsing structured events, and provides robust error handling that distinguishes between recoverable framing/parsing errors and unrecoverable stream errors.

lib/codecs/src/decoding · high confidence

Updated OpenTelemetry Protobuf definitions to v1.0.0

The vendored OpenTelemetry Protocol (OTLP) definitions in lib/opentelemetry-proto have been updated to version 1.0.0. This change refreshes the underlying schema files for traces, metrics, logs, and common resource data, ensuring compatibility with the latest OpenTelemetry specification standards for telemetry data exchange.

lib/opentelemetry-proto/src/proto · high confidence

VRL functions consolidated into a single crate with new categorization and metadata support

All Vector-specific VRL functions (including secret management and semantic meaning assignment) are now centralized in the \lib/vector-vrl/functions\ crate, which serves as the single source of truth for the available function set. This consolidation introduces a new \Category\ enum (Event, Enrichment, Metrics) to classify functions, adds explicit \return\_kind\ and \description\ metadata to parameters, and marks functions like \set\_semantic\_meaning\, \set\_secret\, and \remove\_secret\ as impure to ensure correct evaluation order. Users benefit from a unified function registry that combines standard VRL library functions with Vector-specific extensions, while the \set\_semantic\_meaning\ function now enforces stricter compile-time checks to prevent assigning meanings to non-existent or local fields.

lib/vector-vrl/functions · high confidence

Vector sink rewritten with new configuration style and expanded capabilities

The Vector sink has been rewritten to use the new configurable component style, introducing support for multiple endpoint routing strategies (including load balancing and failover) and configurable HTTP/2 keepalive settings to detect dead connections. It now supports zstd compression in addition to gzip, and enforces a nesting depth limit on events to prevent protobuf encoding issues. The old boolean compression option has been removed in favor of the explicit enum, and the deprecated single \address\ field is deprecated in favor of the new \routing.endpoints\ structure.

src/sinks/vector · high confidence

Test coverage

Add buffer performance benchmarks for disk and memory variants; Add comprehensive benchmark suite for Vector components; Add end-to-end tests for Vector Kubernetes deployment roles; Added Antithesis durability harness and behavior tests for Vector components; Added Antithesis harness tests for end-to-end data conservation and liveness; Added Avro codec test fixtures; Added Avro schema fixture generation tests; Added DNS message parsing benchmarks; Added VRL test harness with documentation, enrichment, and metrics coverage; Added benchmark suite for the rate-limited tracing layer; Added benchmarks for DNS query and update parsing; Added benchmarks for LogEvent::rename\_key performance; Added comprehensive test suite for disk buffer v2; Added comprehensive tests for file watcher behavior and metrics; Added integration tests for vector-config schema generation and macro behavior; Added property-based tests for the disk v2 buffer model; Added protobuf test fixtures and generation tooling; Added schema definition for native encoding tests; Added test coverage for Avro, Native, Native JSON, Protobuf, and Varint framing codecs; Added test coverage for Vector event serialization, size tracking, and iteration; Added test fixtures for vector-core; Added test infrastructure for vector buffers; Added test resources for VRL validation functions; Added tests for metrics label injection and cardinality tracking; Initial RPM package specification for Vector; New benchmark suite for Vector transforms; Reorganized codec benchmark suite into a dedicated subdirectory.

Dependencies

Routine dependency updates across 44 manifests

This release includes routine updates to 44 dependency manifests, covering Rust crates (such as regex, tokio, serde, and aws-sdk), JavaScript packages for the website (including React 19 and TypeScript), and other tooling dependencies. These updates keep the project's dependencies current with upstream releases.

(dependencies) · high confidence

Upgrade website to React 19

The Vector documentation website has been upgraded to React 19. This update modernizes the underlying frontend framework used to render the site, which may require adjustments to any custom React components or third-party libraries integrated into the documentation build process.

website · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 41 → 57 (+15.3)
  • Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

Lenses

  • Code Health 53 → 86 (+32.9)
  • Architecture 69 → 66 (-2.9)
  • Maturity 87 → 84 (-3.4)
  • Readiness 22 → 94 (+72.6)
  • Security 56 → 61 (+5.6)
  • Accessibility 42 (new)

Resolved (135)

  • ADR not followed: Custom DNS resolution removal (website/content/en/highlights/2020-06-18-remove-custom-dns-resolution.md)
  • ADR not followed: Merge existing tcp and udp sources into a single socket source (website/content/en/highlights/2020-01-03-merge-existing-tcp-and-udp-sources-into-a-single-socket-source.md)
  • ADR not followed: The splunk_hec sink does not index fields by default (website/content/en/highlights/2020-01-20-splunk-hec-specify-indexed-fields.md)
  • ADR not followed: Use comma delim server list in kafka sink (website/content/en/highlights/2020-01-12-use-comma-delim-server-list-instead-of-an-array.md)
  • Consequences/trade-offs (e.g. performance cost of handling complex PEMs) are not visible before the clip (website/content/en/highlights/2020-05-27-add-support-for-loading-multiple-cas.md)
  • Consequences/trade-offs (e.g., loss of dynamic encoding flexibility) not visible before clip (website/content/en/highlights/2019-10-21-require-encoding-option-for-console-and-file-sinks.md)
  • Consequences/trade-offs are not present and the body clips mid-sentence inside Getting Started before any trade-offs are stated (website/content/en/highlights/2020-02-28-custom-vector-builds.md)
  • Consequences/trade-offs are not stated (e.g., performance cost of remap expression evaluation vs check_fields) before the decision is framed (website/content/en/highlights/2021-02-16-filter-remap-support.md)
  • Consequences/trade-offs are not visible before the clip (only a one-line limitation is stated) (website/content/en/highlights/2021-01-20-wildcard-identifiers.md)
  • Consequences/trade-offs are not visible before the clip (the body clips mid-## An important step for Vector) and cannot be flagged as missing (website/content/en/highlights/2019-11-21-windows-support.md)
  • Consequences/trade-offs are not visible before the clip marker (the body is clipped mid-table-of-contents) (website/content/en/highlights/2025-01-13-0-44-0-upgrade-guide.md)
  • Consequences/trade-offs of adding these predicates (e.g. performance cost or learning curve) are not present in the visible text (website/content/en/highlights/2020-04-01-more-condition-predicates.md)
  • Context/problem and consequences/trade-offs are absent (only breaking/deprecations lists appear) (website/content/en/highlights/2023-11-07-0-34-0-upgrade-guide.md)
  • Context/problem and consequences/trade-offs are both present but thin; only the decision (breaking change list with migration notes) is explicit and the trade-offs of each change are not detailed (website/content/en/highlights/2022-03-22-0-21-0-upgrade-guide.md)
  • Coverage not measured — test suite did not build
  • Dimension evaluation failed
  • Explicit Decision (the graceful exit behavior) and Consequences/trade-offs are not present in the visible text; only context and future expansion are shown (website/content/en/highlights/2020-05-04-shutdown-vector-if-all-sources-finish.md)
  • Generic title ("0.30 Upgrade Guide") with no context/trade-offs; only one concrete breaking change described in a subsection (website/content/en/highlights/2023-05-22-0-30-0-upgrade-guide.md)
  • High CVE: [GHSA redacted] (Cargo.lock)
  • High IaC: DS-0002 (scripts/cross/Dockerfile)
  • …and 115 more

New (852)

  • Aggregate::flush_event_time_buckets (cognitive 72) (src/transforms/aggregate/event_time.rs)
  • Aggregate::flush_event_time_buckets (cyclomatic 21) (src/transforms/aggregate/event_time.rs)
  • Aggregate::flush_system_time (cognitive 26) (src/transforms/aggregate/transform.rs)
  • Aggregate::record_comparison (cognitive 16) (src/transforms/aggregate/transform.rs)
  • Aggregate::record_comparison_in_map (cognitive 16) (src/transforms/aggregate/event_time.rs)
  • AzureBlobSinkConfig::validate (cognitive 16) (src/sinks/azure_blob/config.rs)
  • AzureBlobSinkConfig::validate (cyclomatic 16) (src/sinks/azure_blob/config.rs)
  • Boundary-crossing change coupling: unix.rs ↔ udp.rs (src/internal_events/unix.rs)
  • BroadcastLayer::on_event (cognitive 17) (src/trace.rs)
  • Builder::load_enrichment_tables (cognitive 18) (src/topology/builder.rs)
  • CI installs an unverified third-party binary (.github/workflows/publish.yml)
  • CI installs an unverified third-party binary (.github/workflows/release_manifests.yml)
  • Change coupling: aws_ecs_metrics.rs ↔ prometheus.rs (src/internal_events/aws_ecs_metrics.rs)
  • Change coupling: file.rs ↔ kubernetes_logs.rs (src/internal_events/file.rs)
  • Change coupling: http.rs ↔ mod.rs (src/components/validation/resources/http.rs)
  • Change coupling: logs.rs ↔ metrics.rs (src/sinks/influxdb/logs.rs)
  • Change coupling: logstash.rs ↔ mod.rs (src/sources/logstash.rs)
  • Change coupling: mod.rs ↔ mod.rs (src/components/validation/resources/mod.rs)
  • Change coupling: mod.rs ↔ mod.rs (src/sources/fluent/mod.rs)
  • Change coupling: mod.rs ↔ syslog.rs (src/sources/fluent/mod.rs)
  • …and 832 more

Changes since last survey

  • 295 commits — 246 feature/other, 49 fixes

By area

  • src/sinks — 47 commits
  • (root) — 45 commits
  • .github/workflows — 29 commits
  • src/sources — 18 commits
  • website/cue — 16 commits
  • vdev/src — 12 commits
  • lib/codecs — 8 commits
  • lib/vector-core — 7 commits
  • distribution/docker — 5 commits
  • lib/vector-buffers — 5 commits
  • lib/vector-config — 5 commits
  • src/transforms — 5 commits
  • .github/dependabot.yml — 4 commits
  • .github/ISSUE_TEMPLATE — 3 commits
  • .github/PULL_REQUEST_TEMPLATE.md — 3 commits
  • lib/vector-config-common — 3 commits
  • scripts/environment — 3 commits
  • src/enrichment_tables — 3 commits
  • src/template — 3 commits
  • website/content — 3 commits

Notable commits

  • fix: chore(deps): bump rustls to 0.23.45 to fix RUSTSEC-2026-0285 (#26376)
  • fix: chore(regression): prefix syslog_loki host label with h- (#26094)
  • fix: chore(website): tailwind v4 visual regression fixes (#26159)
  • fix: fix(aws service): Propagate FIPS endpoint setting to STS AssumeRole clients (#25232)
  • fix: fix(buffers): Stop reporting usage for dropped buffers (#25994)
  • fix: fix(ci): handle prerelease versions in nightly rename and deb packaging (#26398)
  • fix: fix(ci): make awaiting-author label removal idempotent (#26254)
  • fix: fix(ci): replace minio with rustfs in databend integration tests (#26370)
  • fix: fix(codecs): framing.method: varint_length_delimited (#26169)
  • fix: fix(codecs): handle single-chunk GELF messages directly (#26300)
  • fix: fix(codecs): preserve logical types in generated Avro schema fixtures (#26000)
  • fix: fix(codecs): prevent spurious timestamp injection in OTLP trace events (#25404)
  • fix: fix(codecs): three pre-existing chunked_gelf bugs (#26162)
  • fix: fix(config docs): mark required fields explicitly (#26073)
  • fix: fix(config): accept omitted flattened optional internally-tagged enums (#26248)
  • fix: fix(config): avoid schema environment mutation (#26377)
  • fix: fix(databricks_zerobus sink): trim trailing slash from Unity Catalog endpoint (#26194)
  • fix: fix(datadog_agent source)!: decode span links and events from upstream (#26307)
  • fix: fix(datadog_metrics sink): encode resource-prefixed tags as v2 resources (#25973)
  • fix: fix(deps): bump h2 to 0.4.16 and ignore RUSTSEC-2026-0258 for h2 0.3 (#26139)
  • …and 275 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

vectordotdev/vector was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 27 September 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit e44fb91fa1ce822b8ce45d66ea925e497431e6d9 — the exact code this score is about.
  • Scored under rubric-2026.09.15 — the same rubric and the same method as every other entry in this index.
  • Measured by watchdog.canine.dev using codehealth-analyzer preprod-d00c643c3f66.