RyanCodrai/turbovec

Turbovec is a high-performance vector search library that provides a Rust core with Python bindings for storing and querying embeddings. It features a stable v7 index format with crash-safe incremental persistence, supporting 2-bit and 4-bit quantization for efficient memory usage. The system integrates with major LLM frameworks like LangChain and LlamaIndex, offering thread-safe concurrent access and async execution capabilities.

This change introduces the Python binding layer for the TurboQuantIndex vector store, exposing the Rust core to Python via PyO3 and NumPy. The bindings implement strict input validation, replacing opaque panics with descriptive Python exceptions (ValueError/TypeError) for argument mismatches, and introduce a hardened mask-reading mechanism that bypasses Python method lookups to prevent potential filter bypasses from ndarray subclasses. It also includes parallel buffer copying utilities for efficient data ingestion.

turbovec-python/src · high confidence

The \turbovec-python/python\ package is introduced, exposing \IdMapIndex\ and \TurboQuantIndex\ with a new \BATCH\_CHUNK\_SIZE\ default of 4096 rows to make long \search\ and \add\ operations interruptible via Ctrl-C. It includes shared utilities for in-batch duplicate resolution (\\_dedup\), atomic persistence with Windows retry logic (\\_persist\), and configurable similarity modes (\\_similarity\). The package also provides full VectorStore/DocumentStore integrations for LangChain, LlamaIndex, Haystack, and Agno, supporting both cosine and dot-product similarity, thread-safe concurrent access, and async execution off the main event loop.

turbovec-python/python · high confidence

This change introduces a new suite of tools to automate benchmark execution, data handling, and result visualization. The \whm.py\ script provides a weighted harmonic mean scoring harness to evaluate performance across six operations (search, insert, delete, save, load, load\_search) on both ARM and x86 architectures, including single-threaded variants, ensuring that improvements in a target operation do not regress others. The \download\_data.py\ utility manages the acquisition of benchmark datasets (GloVe and OpenAI embeddings) with atomic file writes to prevent corruption. Finally, \create\_diagrams.py\ generates SVG charts in the project's visual style, displaying speed comparisons between TurboQuant and FAISS, as well as recall metrics, to provide clear performance insights for users.

benchmarks · high confidence

The examples directory now includes several new tools to aid in development and performance tuning. The \convert\ example allows users to convert index files between format versions v5, v6, and v7, or to report the version of an existing index. New micro-benchmarks (\insert\_bench\, \load\_bench\_v7\) provide isolated metrics for insertion/removal latency and file load times. Additionally, architecture-specific probes (\kernel\_roofline\, \kernel\_roofline\_neon\, \probe\_2bit\_sdot\, \probe\_2bit\_vnni\, \probe\_2bit\_lutstream\, \stream\_bw\) and cross-arch parity tests (\kernel\_xtest\, \encode\_hash\) are added to measure CPU instruction throughput, memory bandwidth, and ensure deterministic encoding consistency across x86\_64 and aarch64 platforms.

turbovec/examples · high confidence

The \benchmarks/hillclimb\ directory now provides a structured framework for tracking and validating performance optimizations through defined goals and detailed result logs. New goal files (\GOAL\_2bit.md\, \GOAL\_mutate.md\, \GOAL\_persist.md\) establish specific objectives and measurement protocols for 2-bit search, live-index mutation, and whole-file persistence. Accompanying result logs (\LOG.md\, \LOG\_2bit.md\, \LOG\_mutate.md\, \LOG\_persist.md\, \LOG\_search.md\, \LOG\_sync.md\) document the outcomes of hill-climbing experiments, including hypothesis testing, baseline comparisons, and correctness verification across ARM and x86 architectures.

benchmarks/hillclimb · high confidence

The first stable release (1.0.0) establishes v7 as the only on-disk format for both the Rust crate and Python package, refusing to load older v5/v6 files to ensure format stability. It introduces sync() for incremental, crash-safe saves that write only changed rows, and provides a new convert utility to migrate existing v5 or v6 index files forward to v7 or back. This release also includes several critical bug fixes, such as resolving aarch64 search performance regressions, preventing unbounded memory allocation during load, and fixing codes buffer doubling on small adds.

(repo-wide) · high confidence

The library has been restructured into a pure Rust core crate (\turbovec/src\) with Python bindings, introducing the v7 index format as the native read/write standard. This format uses an incremental, headerized slot-array container with atomic save protocols and CRC-32C integrity checks. To support existing data, a new \convert\ module allows seamless migration between v5, v6, and v7 index files in any direction without re-encoding. The core also introduces stable external IDs via \IdMapIndex\, robust input validation that returns descriptive errors instead of panicking, and cross-platform reproducible quantization codebooks.

turbovec/src · high confidence

Users can now run \cargo build\ directly on macOS without encountering symbol resolution failures. A new \build.rs\ script was added to automatically emit the correct linker arguments (specifically \-undefined dynamic\_lookup\) required for Python extension modules, ensuring that symbols from the Python interpreter resolve correctly at load time.

turbovec-python · high confidence

Added adversarial durability and performance regression tests; Added downstream smoke test to verify public API stability; Added thread-safety tests for integration stores; Comprehensive benchmark suite for TurboQuant recall, speed, and compression; Expanded test coverage for Python integration stores and core bindings.

This change introduces the initial 1.0.0 release of the turbovec vector search library, establishing a Rust workspace that includes the core \turbovec\ crate and its \turbovec-python\ bindings. The update pins specific dependency versions to ensure deterministic encoding and index stability: \rand\_chacha\ is fixed at 0.3.1 to prevent silent index corruption from stream changes, and \statrs\ is pinned at 0.17.1 to maintain consistent calibration anchors. The Python bindings require Rust 1.89 due to stabilized AVX-512 intrinsics and declare optional integrations for LangChain, LlamaIndex, Haystack, and Agno.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 73 → 74 (+1.6)
• Rubric changed (rubric-2026.09.8 → rubric-2026.09.16) — scores are not directly comparable.

• Code Health 79 → 79 (-0.2)
• Architecture 100 → 95 (-4.9)
• Maturity 68 → 67 (-0.6)
• Readiness 73 → 76 (+3.0)
• Security 76 → 83 (+6.8)
• Performance 100 (new)

• Duplicated block (13 lines × 2) (turbovec-python/python/turbovec/agno.py)
• Duplicated block (16–19 lines × 2) (turbovec-python/python/turbovec/agno.py)
• Duplicated block (17–18 lines × 2) (turbovec-python/python/turbovec/langchain.py)
• Duplicated block (18 lines × 2) (turbovec-python/python/turbovec/haystack.py)
• Duplicated block (21–24 lines × 2) (turbovec-python/python/turbovec/langchain.py)
• Duplicated block (5 lines × 2) (turbovec-python/python/turbovec/_persist.py)
• Duplicated block (8 lines × 2) (turbovec-python/python/turbovec/agno.py)
• Duplicated block (8 lines × 2) (turbovec-python/python/turbovec/langchain.py)
• Duplicated block (8–13 lines × 2) (turbovec-python/python/turbovec/agno.py)
• Hotspot: benchmarks/hillclimb/bench_ops.py (benchmarks/hillclimb/bench_ops.py)
• Hotspot: turbovec-python/python/turbovec/agno.py (turbovec-python/python/turbovec/agno.py)
• Hotspot: turbovec-python/python/turbovec/haystack.py (turbovec-python/python/turbovec/haystack.py)
• Hotspot: turbovec-python/python/turbovec/langchain.py (turbovec-python/python/turbovec/langchain.py)
• Hotspot: turbovec-python/python/turbovec/llama_index.py (turbovec-python/python/turbovec/llama_index.py)
• Off the main sequence: turbovec
• Outdated: libc
• Outdated: pyo3
• Outdated: pyo3-build-config
• Repeated repair: turbovec-python/python/turbovec/_persist.py (turbovec-python/python/turbovec/_persist.py)

• Containers 0 added · 0 removed · contexts 0 added · 1 removed · edges 0 added · 0 removed

• python

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

RyanCodrai/turbovec was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.