pgvector/pgvector-elixir

This is an Elixir library that provides Ecto and Postgrex integrations for storing and querying vector data in PostgreSQL using the pgvector extension. It supports multiple vector types, including dense, half-precision, sparse, and binary vectors, and exposes distance functions for similarity search. The system also includes example scripts demonstrating integration with various embedding providers and search strategies.

5 changes.

This period marks the initial release of the pgvector Elixir library, introducing core support for vector similarity search in PostgreSQL. The work established the foundational Pgvector struct, Postgrex extensions for automatic type encoding, and comprehensive Ecto integration with distance-based query functions. A full test suite was also implemented to verify type conversions and database interactions.

8 changes.

This period focused on expanding pgvector's core capabilities by adding support for half-precision and sparse vector types. It also significantly broadened the library's practical applicability by introducing a diverse suite of examples covering various embedding providers, hybrid and sparse search strategies, and distributed search with Citus.

A new example demonstrating how to use Bumblebee with pgvector for semantic search. The example configures the EXLA backend and provides a script that loads the 'all-MiniLM-L6-v2' model, generates embeddings for sample text, stores them in a PostgreSQL database with the vector extension, and performs a similarity search using the cosine distance operator.

examples/bumblebee · high confidence

A new example script (examples/citus/example.exs) demonstrates how to use pgvector with Citus for distributed vector similarity search. The script generates synthetic embedding data, configures the database with necessary extensions (citus, vector) and GUC settings, creates a distributed table partitioned by category, loads data in parallel using binary COPY, builds an HNSW index, and executes distributed nearest-neighbor queries.

examples/citus · high confidence

A new example script demonstrates how to integrate the Cohere API for generating binary vector embeddings and storing them in a PostgreSQL database using the pgvector extension. The script shows the full workflow: configuring Postgrex with custom types, creating a table with a bit(1536) embedding column, calling the Cohere embed-v4.0 model to produce ubinary embeddings, inserting them into the database, and performing a similarity search using the \<\~\> operator.

examples/cohere · high confidence

The examples/openai directory now includes a new example script (example.exs) that demonstrates how to generate text embeddings using the OpenAI text-embedding-3-small model and store them in a PostgreSQL database using the pgvector extension. The example shows how to create a vector table, insert document embeddings, and perform similarity searches using the cosine distance operator.

examples/openai · high confidence

New Postgrex extension modules have been added to support the pgvector data types (vector, halfvec, and sparsevec). These extensions enable automatic encoding and decoding of these vector types when using Postgrex, allowing seamless integration of vector data in database queries without manual binary handling.

lib/pgvector/extensions · high confidence

The examples/loading directory now includes a new example script (example.exs) that demonstrates how to bulk load 100,000 vector embeddings into a PostgreSQL database using the Pgvector extension. The example shows how to generate random embeddings, establish a connection with custom Postgrex types, and use the PostgreSQL COPY command in binary format for efficient data ingestion, including a configurable timeout of 30 seconds. It also outlines the optional steps for creating HNSW indexes and updating planner statistics after the initial load.

examples/loading · high confidence

A new example demonstrating hybrid search capabilities has been added to the examples/hybrid\_search directory. This example combines semantic search using the 'sentence-transformers/multi-qa-MiniLM-L6-cos-v1' model via Bumblebee with keyword search in PostgreSQL using pgvector. It illustrates how to embed text, store vectors in a PostgreSQL database, and perform a combined search using Reciprocal Rank Fusion (RRF) to rank results based on both semantic similarity and keyword relevance.

_examples/hybrid\search · high confidence

The examples/sparse\_search directory now includes a complete demonstration of sparse semantic search. The example configures the EXLA backend for computation and uses the Bumblebee library to load the 'opensearch-neural-sparse-encoding-v1' model and tokenizer. It shows how to generate sparse vector embeddings for text documents, store them in a PostgreSQL database using the pgvector extension (specifically the sparsevec type), and perform similarity searches to retrieve relevant content based on a query.

_examples/sparse\search · high confidence

The library now includes new \Pgvector.HalfVector\ and \Pgvector.SparseVector\ modules, enabling users to create, manipulate, and inspect vectors using 16-bit floating-point precision and sparse data structures respectively. These additions allow for more memory-efficient storage and computation when working with half-precision tensors (via Nx) or sparse datasets, expanding the range of vector types supported by pgvector.

lib/pgvector · high confidence

This change introduces Ecto support for the pgvector library, allowing users to store and query vector data directly within Ecto schemas. New Ecto types are provided for \:vector\, \:halfvec\, \:sparsevec\, and \:bit\ data types, enabling seamless casting, loading, and dumping of these values. Additionally, a new \Pgvector.Ecto.Query\ module exposes macro-based distance functions (L2, cosine, L1, inner product, Hamming, and Jaccard) that can be used in Ecto queries to perform similarity searches against the database.

lib/pgvector/ecto · high confidence

The library now provides a core \Pgvector\ struct to represent vector data, along with functions to create vectors from lists, Nx tensors (when Nx is loaded), and binary representations, as well as to convert them back to lists, tensors, or binary. It also includes support for \HalfVector\ and \SparseVector\ types, and exposes an \extensions/0\ function that registers Postgrex extensions for Vector, Halfvec, and Sparsevec types.

lib · high confidence

Added new test files covering the core Pgvector types (Vector, HalfVector, SparseVector, Bit) and their integrations with Ecto and Postgrex. The tests verify type creation, conversion to/from lists and Nx tensors, equality checks, and distance-based ordering (L2, cosine, L1, Hamming, Jaccard) within Ecto schemas and raw Postgrex queries.

test · high confidence

This change introduces the initial version (0.4.1) of the pgvector Elixir library, providing support for vector similarity search in PostgreSQL. The package requires Postgrex and optionally supports Ecto and Nx (version 0.5 or 1.0). Alongside the core library, several example projects are added to demonstrate usage with Bumblebee, Citus, Cohere, OpenAI, hybrid search, sparse search, and bulk loading, each with their own dependency configurations.

(dependencies) · high confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• First survey — no prior run to compare against. CAI 65.

• Code Health 97
• Architecture 69
• Maturity 55
• Readiness 63
• Security 100

• 178 commits — 169 feature/other, 9 fixes

• (root) — 70 commits
• lib/pgvector — 34 commits
• .github/workflows — 23 commits
• test/postgrex_test.exs — 11 commits
• examples/loading — 6 commits
• test/ecto_test.exs — 6 commits
• examples/cohere — 5 commits
• lib/pgvector.ex — 4 commits
• test/sparse_vector_test.exs — 4 commits
• examples/bumblebee — 3 commits
• examples/hybrid_search — 3 commits
• examples/sparse_search — 3 commits
• examples/citus — 2 commits
• examples/openai — 2 commits
• postgrex/example.exs — 2 commits

• fix: Fixed CI
• fix: Fixed CI
• fix: Fixed CI
• fix: Fixed error with postgrex only
• fix: Fixed example [skip ci]
• fix: Fixed link [skip ci]
• fix: Fixed test warning [skip ci]
• fix: Fixed warning when Nx not loaded
• fix: Moved example and fixed CI
• change: Add type definition to Vector Ecto type (#11)
• change: Added Bumblebee example [skip ci]
• change: Added Citus example [skip ci]
• change: Added Cohere example [skip ci]
• change: Added HNSW to docs [skip ci]
• change: Added OpenAI embeddings example [skip ci]
• change: Added Pgvector struct - closes #6
• change: Added Pgvector.extensions/0 function
• change: Added Postgrex example
• change: Added bulk loading example [skip ci]
• change: Added check for rank
• …and 158 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

pgvector/pgvector-elixir was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.