stanfordnlp/dspy
66.7
Adequate · 14 November 2026
35.2k
lines of production code
Python
primary language
7
measurements over time
What this system is
Features
Automated tooling to vendor lm15 and sync LiteLLM metadata
Added two new scripts to automate the maintenance of bundled dependencies. \scripts/update\_vendored\_lm15.py\ manages the \lm15\ package as a Git subtree within \dspy/\_vendor\, handling version pinning, contract verification, and license storage. \scripts/update\_model\_metadata.py\ fetches the latest LiteLLM wheel to extract and compress its model pricing and context window data into a local snapshot, ensuring the runtime loader has an up-to-date fallback map without requiring a direct import of the upstream library.
scripts · high confidence
Initialize project structure with pre-commit hooks, contribution guide, license, and security policy
The repository now includes a \.pre-commit-config.yaml\ file that enforces code quality using \ruff\ for linting and formatting on Python files within the \dspy\ and \tests\ directories. A \CONTRIBUTING.md\ guide has been added to instruct developers on the fork-and-PR workflow, environment setup via \uv\ or \conda\, and policies for AI-assisted contributions. Additionally, a \LICENSE\ file (MIT) and a \SECURITY.md\ file defining vulnerability reporting procedures are now present, establishing the legal and security framework for the project.
(repo-wide) · high confidence
Removals
Removal of dsp.primitives module
The \dsp.primitives\ package has been removed, deleting the \\_\init\\_.py\ and all its submodules (\demonstrate\, \predict\, \primitives\, \search\). This eliminates the legacy implementation of demonstration handling, prediction generation, and retrieval primitives that were previously exposed through this namespace.
dsp/primitives · high confidence
Removal of legacy DSP modules and caching utilities
The \dsp/modules\ package has removed the legacy \GPT3\ class, the \ColBERTv2\ retrieval module, and the \cache\_utils\ infrastructure. Users relying on these specific components for OpenAI text completion, ColBERT-based retrieval, or the joblib-based caching mechanism will no longer have access to them through this module path, as the files and their exports have been deleted.
dsp/modules · high confidence
Removal of legacy DSP template modules
The \dsp/templates\ package has been completely removed, deleting the \\_\init\\_.py\ entry point and the underlying implementation files (\template\_v2.py\, \template\_v3.py\, and \utils.py\). This eliminates the legacy \TemplateV2\ and \Template\ classes, along with helper functions like \passages2text\ and \format\_answers\, meaning users can no longer import or utilize these specific template structures for prompt formatting.
dsp/templates · high confidence
Removal of legacy dsp module exports
The dsp package no longer exposes its internal modules, primitives, templates, or settings via the main \_\init\\_.py file. This change removes the automatic re-export of these components, meaning users can no longer import them directly from the top-level dsp namespace and must instead import them from their specific submodules.
dsp · high confidence
Removal of legacy dsp.utils module
The \dsp/utils\ package has been removed, deleting the \\_\init\\_.py\, \metrics.py\, \settings.py\, and \utils.py\ files. This eliminates legacy utilities including the singleton \Settings\ context manager, text normalization and evaluation metrics (such as EM, F1, and HotPotF1), and general helper functions like \dotdict\ and \print\_message\. Users relying on these specific legacy components will need to migrate to the updated DSPy equivalents or external libraries.
dsp/utils · high confidence
Removal of legacy evaluation utility
The \dsp/evaluation/utils.py\ module has been removed. This deletion eliminates the legacy \evaluate\ function that previously executed a model against a dataset, printed summary statistics, and displayed results in an IPython environment. Users relying on this specific utility for evaluation workflows will need to adopt alternative methods.
dsp/evaluation · high confidence
Removal of legacy v1 joblib cache wrappers
The legacy v1 joblib cache wrapper files for ColbertV2 and GPT3 modules have been removed. This cleanup eliminates the deprecated caching mechanism that previously wrapped request functions with \@NotebookCacheMemory.cache\ and \@functools.lru\_cache\, aligning the codebase with the v2 release standards.
cache/joblib/dsp/modules/colbertv2 · high confidence
Behavioural changes
DSPy 3.4.0 release with vendorized lm15 provider layer
This entry covers the DSPy 3.4.0 release, which introduces a new vendorized \lm15\ package (version 1.0.0rc3) to handle low-level, provider-neutral LLM API interactions. The \dspy/\_\init\\_.py\ module is restructured to expose this new layer alongside updated adapters (\ChatAdapter\, \JSONAdapter\, \XMLAdapter\, \TwoStepAdapter\), new data types (\Image\, \Audio\, \File\, \Code\, \Reasoning\), and utility functions (\asyncify\, \syncify\, \streamify\, \track\_usage\). The release also includes a comprehensive set of new error classes for better LM error handling and updates to the settings configuration API.
dspy · high confidence
Test coverage
14 commits adding/updating tests in tests/docs; 17 commits adding/updating tests in tests/flex; Added GEPA test fixtures for dummy LM and custom component selection; Added MCP test server resource for tool testing; Added comprehensive test suite for DSPy primitives; Added comprehensive test suite for DSPy utility modules; Added static typing tests for asyncify, callbacks, and parallelizer; Added test coverage for decision types, aggregation, and interpreter-based prediction modules; Added test coverage for evaluation metrics and engine behavior; Added test infrastructure for DSPy reliability, interpreters, and vendored lm15; Added test infrastructure for LiteLLM server integration testing; Added tests for ColBERTv2 error handling and Embeddings save/load functionality; Added tests for Dataset seed reset and CSV input handling; Added tests for File, Image, and custom type handling in signatures; Added tests for GroundedProposer instruction generation; Added tests for package metadata; Added tests for streaming ReActV2 tool calls and status message providers; Added tests for the callback system; Added tests for the experimental ReAnchor calibration optimizer; Expanded test coverage for DSPy client infrastructure and safety features; Expanded test coverage for adapter infrastructure and type handling; Initial DSPy reliability test suite and generation tooling; Tests verify retirement of legacy LM types and migration to lm15.
Dependencies
DSPy 3.4.0 release and documentation build updates
The main package is updated to version 3.4.0 with a Python requirement of 3.10–3.14 and new core dependencies including pydantic\>=2.11.0, openai\>=1.66.2, litellm\>=1.65.8, and gepa\[dspy\]==0.1.4. A compatibility alias package (dspy-ai) is added to redirect users to the main dspy package. Documentation tooling is updated with mkdocstrings 1.0.6, mkdocstrings-python 2.0.7, and mistune 3.3.4, alongside a new docs/requirements.txt and Pipfile for building the site.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 59 → 67 (+8.2)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 62 → 62 (-0.4)
- Architecture 75 → 79 (+3.7)
- Maturity 63 → 71 (+7.4)
- Readiness 50 → 66 (+16.2)
- Security 70 → 75 (+5.1)
- Performance 100 → 100 (+0.0)
Resolved (69)
- Adapter.format_conversation_history (cognitive 53) (dspy/adapters/base.py)
- Adapter.format_conversation_history (cyclomatic 26) (dspy/adapters/base.py)
- Duplicated block (12 lines × 2) (dspy/predict/react.py)
- Duplicated block (12 lines × 2) (dspy/predict/react_v2.py)
- Duplicated block (12–17 lines × 2) (dspy/clients/execution.py)
- Duplicated block (14 lines × 2) (dspy/predict/refine.py)
- Duplicated block (16 lines × 2) (dspy/predict/react_v2.py)
- Duplicated block (19 lines × 2) (dspy/predict/react_v2.py)
- Duplicated block (19 lines × 2) (dspy/predict/rlm.py)
- Duplicated block (28 lines × 2) (dspy/predict/rlm.py)
- Duplicated block (5 lines × 2) (dspy/adapters/base.py)
- Duplicated block (5 lines × 2) (dspy/predict/predict.py)
- Duplicated block (7 lines × 2) (dspy/clients/execution.py)
- Duplicated block (8 lines × 2) (dspy/clients/lm.py)
- Duplicated block (8 lines × 2) (dspy/predict/react_v2.py)
- Duplicated block (8 lines × 2) (dspy/predict/react_v2.py)
- Duplicated block (8 lines × 2) (dspy/primitives/python_interpreter.py)
- Duplicated block (9 lines × 2) (dspy/adapters/types/tool.py)
- Duplicated block (9 lines × 2) (dspy/predict/react_v2.py)
- Duplicated block (9 lines × 2) (dspy/primitives/base_module.py)
- …and 49 more
New (104)
- BaseLM.copy (cognitive 16) (dspy/clients/base_lm.py)
- BaseLM.load_state (cognitive 16) (dspy/clients/base_lm.py)
- BridgeRuntime._to_prediction (cognitive 18) (dspy/predict/flex/bridge.py)
- Critical CVE: [GHSA redacted] (uv.lock)
- Dependency hygiene PARTLY measured — Python dependencies read, no exact pin to grade for currency
- Duplicated block (10 lines × 2) (dspy/utils/callback.py)
- Duplicated block (11 lines × 2) (dspy/predict/refine.py)
- Duplicated block (12 lines × 2) (dspy/predict/react.py)
- Duplicated block (15 lines × 2) (dspy/predict/rlm.py)
- Duplicated block (16 lines × 2) (dspy/predict/react.py)
- Duplicated block (18 lines × 2) (dspy/predict/rlm.py)
- Duplicated block (19 lines × 2) (dspy/predict/react.py)
- Duplicated block (25 lines × 2) (dspy/predict/rlm.py)
- Duplicated block (50 lines × 2) (dspy/predict/rlm.py)
- Duplicated block (8 lines × 2) (dspy/predict/react.py)
- Duplicated block (8 lines × 2) (dspy/predict/react.py)
- Duplicated block (9 lines × 2) (dspy/predict/react.py)
- Edited copy of a member (11 corresponding lines) (dspy/predict/predict.py)
- Edited copy of a member (13 corresponding lines) (dspy/adapters/chat_adapter.py)
- Edited copy of a member (13 corresponding lines) (dspy/clients/databricks.py)
- …and 84 more
Changes since last survey
- 76 commits — 52 feature/other, 24 fixes
By area
- docs/docs — 17 commits
- dspy/primitives — 12 commits
- dspy/clients — 11 commits
- dspy/adapters — 10 commits
- (root) — 5 commits
- tests/clients — 4 commits
- dspy/predict — 3 commits
- dspy/utils — 3 commits
- docs/scripts — 2 commits
- dspy/_vendor — 2 commits
- dspy/teleprompt — 2 commits
- dspy/datasets — 1 commit
- dspy/signatures — 1 commit
- tests/adapters — 1 commit
- tests/primitives — 1 commit
- tests/utils — 1 commit
Notable commits
- fix: Fix tool call IDs in serialized ReAct history (#10844)
- fix: Fix two-step bounding box test output formatting (#10728)
- fix: fix(adapters): preserve nested XML enum and null guidance (#10859)
- fix: fix(cache): handle restricted disk entries consistently (#10858)
- fix: fix(clients): exclude LiteLLM releases with eager MCP imports (#10863)
- fix: fix(dspy): await async tools under RLM.acall in PythonInterpreter (#10808)
- fix: fix(dspy): find dspy's distribution by the package it installs (#10864)
- fix: fix(dspy): let majority() vote on non-string field values (#10720)
- fix: fix(dspy): parse negative numbers in string signature Literal types (#10678)
- fix: fix(dspy): preserve integer precision when loading pandas data (#10471)
- fix: fix(dspy): support class-based signatures in Flex (#10768)
- fix: fix(interpreter): retry transient sandbox download failures (#10817)
- fix: fix(rlm): render REPL code and output as plain text in prompts (#10812)
- fix: fix(teleprompt): respect MIPROv2.compile seed=0 (#10322)
- fix: fix(tool): keep nested schema references scoped in native declarations (#10845)
- fix: fix(tool): validate RootModel leaves in schema-compatible JSON form (#10846)
- fix: fix(typing): preserve types of experimental classes and properties (#10657)
- fix: fix(xml): honor positional tuple schemas (#10850)
- fix: fix(xml): parse nested numeric and boolean literals (#10840)
- fix: fix: honor reasoning dictionaries in ChainOfThought (#10842)
- …and 56 more
Architecture
- Unchanged — 0 containers · 1 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
stanfordnlp/dspy was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 2c07ebbee3a66f6e04b71e141c6c198f1b31a129 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-36d4cd83012f.