Skip to content
CAI
Software that uses CAICheck a score

huggingface/smolagents

72.8

Strong · 16 November 2026

15.1k

lines of production code

Python

primary language

5

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Features

Add Smolagents Chat Server Demo

A new example server has been added to the examples/server directory, providing a web-based chat interface for interacting with an AI code agent. The demo uses the Qwen/Qwen3-Next-80B-A3B-Thinking model and integrates MCP tools via MCPClient, accessible at http://localhost:8000.

examples/server · high confidence

Introduce Open Deep Research example with multi-agent web research capabilities

Adds a new example in \examples/open\_deep\_research\ that replicates OpenAI's Deep Research workflow. The example provides a \run.py\ script and a \GradioUI\ app (\app.py\) for users to run a multi-agent system consisting of a manager \CodeAgent\ and a managed \ToolCallingAgent\ specialized in web browsing. It includes a \run\_gaia.py\ script for evaluating the agent on the GAIA benchmark, an \analysis.ipynb\ notebook for inspecting results, and a \visual\_vs\_text\_browser.ipynb\ notebook for comparing browser implementations. The setup requires installing dependencies from \requirements.txt\ and setting environment variables for API keys (e.g., \SERPAPI\_API\_KEY\, \OPENAI\_API\_KEY\).

_examples/open\_deep\research · high confidence

Introduce smolagents v1.27.0.dev0 library

This change introduces the smolagents library (version 1.27.0.dev0), providing a framework for building and running AI agents. It includes core agent classes (CodeAgent, ToolCallingAgent), a local Python executor with security restrictions, a Gradio UI for interactive agent demos, and a CLI for quick setup. The library also supports remote execution via Docker, E2B, and Modal, integrates with various LLM providers (OpenAI, Hugging Face, LiteLLM), and includes tools for web search, Python code execution, and MCP server connections.

src/smolagents · high confidence

Introduces Open Deep Research example with multi-modal agent tools

Adds a new example application in \examples/open\_deep\_research/scripts\ that enables agents to perform deep research tasks. This includes a text-based web browser (\text\_web\_browser.py\) for navigating and searching the web, a document conversion utility (\mdconvert.py\) for handling various file formats, and specific tools for visual question answering (\visual\_qa.py\) and text inspection (\text\_inspector\_tool.py\). The package also provides scripts for running agent workflows (\run\_agents.py\), scoring GAIA benchmark results (\gaia\_scorer.py\), and managing browser cookies (\cookies.py\).

_examples/open\_deep\research/scripts · high confidence

New Plan Customization example with Human-in-the-Loop support

Added a new example in \examples/plan\_customization\ that demonstrates how to implement Human-in-the-Loop strategies for agent planning. The example shows how to use step callbacks to interrupt execution after a \PlanningStep\, allowing users to review, approve, modify, or cancel the agent's plan in real-time. It also illustrates how to preserve agent memory across multiple runs using \reset=False\ to resume execution with the modified plan.

_examples/plan\customization · high confidence

New async agent example using Starlette and CodeAgent

Added a new example in the \examples/async\_agent\ directory demonstrating how to integrate a synchronous \CodeAgent\ from the \smolagents\ library into an asynchronous Starlette web application. The example shows how to use \anyio.to\_thread.run\_sync\ to offload the blocking agent execution to a background thread, preventing it from blocking the async event loop, and exposes a \/run-agent\ POST endpoint for task submission.

_examples/async\agent · high confidence

New example scripts for agents, RAG, and execution environments

Added a suite of new example scripts in the examples directory demonstrating key capabilities: agent configuration with various LLM providers (InferenceClient, LiteLLM, OpenAI, Ollama) in agent\_from\_any\_llm.py and multi\_llm\_agent.py; Retrieval-Augmented Generation (RAG) using lexical search (BM25) in rag.py and semantic search (ChromaDB) in rag\_using\_chromadb.py; structured outputs via MCP in structured\_output\_tool.py; SQL querying in text\_to\_sql.py; sandboxed code execution with Blaxel, Docker, E2B, and Modal in sandboxed\_execution.py; a Gradio UI for agent interaction in gradio\_ui.py; and multi-agent orchestration with instrumentation in inspect\_multiagent\_run.py.

examples · high confidence

New smolagents benchmark runner and scoring notebook

Added a new Python script (\run.py\) and Jupyter notebook (\score.ipynb\) to the \examples/smolagents\_benchmark\ directory. The runner script allows users to execute agent evaluations on the \smolagents/benchmark-v1\ dataset using \CodeAgent\ or \ToolCallingAgent\ architectures with either \InferenceClientModel\ or \LiteLLMModel\ backends, supporting parallel execution and result export. The accompanying notebook provides utilities to load, normalize, and score these answers against ground truth for GAIA, MATH, and SimpleQA tasks.

_examples/smolagents\benchmark · high confidence

Test coverage

359 commits adding/updating tests in tests

Test coverage in tests — 359 commits (109 fixs), 39 files.

tests · high confidence · unverified

Dependencies

Initial release of smolagents with structured dependency management

The smolagents library is introduced as a barebones Python library for building agents that write code to call tools or orchestrate other agents. The package defines a core set of dependencies including huggingface-hub, requests, rich, jinja2, pillow, and python-dotenv, while organizing optional capabilities into extras such as bedrock, blaxel, docker, e2b, gradio, litellm, mcp, mlx-lm, modal, openai, telemetry, toolkit, transformers, vision, and vllm. Example projects for async agents and open deep research are provided with their own specific requirements files to support distinct use cases.

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 75 → 73 (-2.4)
  • Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.

Lenses

  • Code Health 69 → 65 (-4.2)
  • Architecture 100 → 100 (+0.0)
  • Maturity 75 → 73 (-1.8)
  • Readiness 75 → 75 (+0.0)
  • Security 85 → 85 (+0.0)
  • Domain Modelling 100 → 100 (+0.0)
  • Performance 100 → 100 (+0.0)

Resolved (1)

  • No ADRs found

New (18)

  • Edited copies of one member (3 members, 22+ corresponding lines) (examples/multiple_tools.py)
  • Edited copies of one member (3 members, 25+ corresponding lines) (src/smolagents/models.py)
  • Edited copies of one member (3 members, 8+ corresponding lines) (src/smolagents/tool_validation.py)
  • Edited copy of a member (26 corresponding lines) (src/smolagents/local_python_executor.py)
  • Edited copy of a member (31 corresponding lines) (src/smolagents/mcp_client.py)
  • Edited copy of a member (4 corresponding lines) (examples/open_deep_research/scripts/run_agents.py)
  • Edited copy of a member (42 corresponding lines) (examples/open_deep_research/scripts/text_inspector_tool.py)
  • Edited copy of a member (45 corresponding lines) (src/smolagents/models.py)
  • Edited copy of a member (5 corresponding lines) (examples/rag.py)
  • High CVE: [GHSA redacted] (examples/open_deep_research/requirements.txt)
  • Medium CVE: [GHSA redacted] (examples/open_deep_research/requirements.txt)
  • Medium CVE: PYSEC-2025-211 (examples/open_deep_research/requirements.txt)
  • No ADRs found (pyproject.toml)
  • Orphaned file (src/smolagents/_function_type_hints_utils.py)
  • Orphaned file (src/smolagents/monitoring.py)
  • Orphaned file (src/smolagents/tool_validation.py)
  • Orphaned file (src/smolagents/vision_web_browser.py)
  • Orphaned knowledge (src/smolagents/serialization.py)

Changes since last survey

  • 9 commits — 6 feature/other, 3 fixes

By area

  • (root) — 4 commits
  • .github/workflows — 3 commits
  • tests/test_agents.py — 1 commit
  • tests/test_local_python_executor.py — 1 commit

Notable commits

  • fix: Fix MCP extra installation with incompatible SDK 2.x (#2585)
  • fix: Fix flaky test_transformers_toolcalling_agent (#3180)
  • fix: Fix pandas exception test for pandas 3 error message (#3176)
  • change: Add workflow to close pull requests without a validated and assigned issue (#3183)
  • change: Ask agents to run make style before committing (#3185)
  • change: Bump the actions group across 1 directory with 2 updates (#3147)
  • change: Docs: build English only, other languages are machine-translated (#3091)
  • change: Require an assigned issue before opening a pull request (#3178)
  • change: Require issues to be validated and assigned before opening a pull request (#3182)

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

huggingface/smolagents was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 16 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit e1249d373d58db6abc73882ee394f6bb3854c89f — the exact code this score is about.
  • Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.73+preprod-52de90b74216.