Skip to content
CAI
Software that uses CAICheck a score

QwenLM/Qwen

43.1

Weak · 16 November 2026

6k

lines of production code

Python

primary language

5

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

This system is an open-source framework for deploying, fine-tuning, and evaluating the Qwen family of large language models. It provides tools for inference across various hardware accelerators, including NVIDIA GPUs, Ascend NPUs, and Hygon DCUs, alongside scripts for full, LoRA, and QLoRA fine-tuning. The codebase also includes evaluation benchmarks, function-calling examples, and containerized deployment options to facilitate model adaptation and integration.

Features

Add chat-mode evaluation scripts and update existing benchmarks for Qwen-7B-Chat

The eval directory now includes new zero-shot evaluation scripts for the chat model (Qwen-7B-Chat) across C-Eval, MMLU, GSM8K, and HumanEval, alongside a new script for tool-use plugin evaluation. Existing non-chat evaluation scripts (C-Eval, GSM8K, HumanEval) have been updated to support batched inference, proper attention masks, and explicit padding token configuration, while the documentation has been updated to reflect the new chat-specific reproduction commands.

eval · high confidence

Add inference support for Qwen models on Ascend 910 and Hygon DCU hardware

Users can now run Qwen models on Ascend 910 and Hygon DCU accelerators. The \ascend-support\ directory provides Docker-based inference using MindSpore and MindFormers, including scripts to convert and run the Qwen-7B-Chat model. The \dcu-support\ directory adds Hygon DCU compatibility via the fastllm framework, offering CLI and web-based demos, along with tools to convert Hugging Face Qwen models into the fastllm format for optimized inference.

ascend-support, dcu-support · high confidence

Added finetuning scripts and DeepSpeed configurations

The finetune directory now includes shell scripts and configuration files to support various training modes. Users can run distributed training with DeepSpeed ZeRO-2 or ZeRO-3 using the new \ds\_config\_zero2.json\ and \ds\_config\_zero3.json\ files, or use single-GPU scripts. The provided scripts cover full finetuning, LoRA finetuning, and QLoRA finetuning (using 4-bit quantized models), with gradient checkpointing enabled to manage memory usage.

finetune · high confidence

Initial Docker support for Qwen-Chat

This change introduces Docker support for running Qwen-Chat, providing three CUDA-specific Dockerfiles (CUDA 11.4, 11.7, and 12.1) and helper shell scripts to easily launch the CLI demo, web demo, and OpenAI-compatible API in containers. The Dockerfiles bundle necessary dependencies like PyTorch, Flash Attention, and fine-tuning libraries, while the scripts handle image pulling, checkpoint mounting, and container startup with appropriate GPU and port configurations.

docker · high confidence

Initial release of Qwen models and documentation

This commit introduces the initial codebase and documentation for the Qwen series of large language models, including Qwen-1.8B, Qwen-7B, Qwen-14B, and Qwen-72B, as well as their chat-aligned counterparts (Qwen-Chat). It provides the necessary Python scripts for inference (\cli\_demo.py\, \web\_demo.py\), fine-tuning (\finetune.py\), and an OpenAI-compatible API server (\openai\_api.py\). The release includes comprehensive documentation in English, Chinese, Japanese, French, and Spanish, covering installation, usage, performance benchmarks, and licensing, along with an FAQ to assist users with common setup issues such as Flash Attention installation and tokenizer configuration.

(repo-wide) · high confidence

Initial release of Qwen recipe notebooks and Ascend NPU fine-tuning guide

This update introduces a comprehensive set of Jupyter notebook recipes for the Qwen model family, providing step-by-step guides for various use cases. The collection includes application demos such as a web-based chatbot using Gradio and a retrieval-augmented Q&A system with LangChain. It also covers fine-tuning workflows, offering tutorials for full-parameter, LoRA, and QLoRA adaptation on both single and multiple GPUs using DeepSpeed, as well as a specific guide for fine-tuning on Ascend NPUs.

recipes · high confidence

New examples for function calling, code auto-commenting, and vocabulary expansion

The examples directory now includes several new scripts and documentation. function\_call\_examples.py and function\_call\_finetune\_examples.py demonstrate how to use the model for tool use (function calling) via both an OpenAI-compatible API and ReAct-style prompting, including guidance for fine-tuning. auto\_comments.py and its documentation show how to automatically generate Chinese comments for Python code files or folders. add\_merges.py provides a utility to expand the tokenizer vocabulary by learning new BPE merges from a frequency list. Additionally, system\_prompt.md explains how to use system prompts for role-playing and behavior customization, and langchain\_tooluse.ipnb shows how to integrate LangChain tools like Google Search and WolframAlpha using ReAct prompting.

examples · high confidence

Dependencies

Initial dependency manifests for core, web demo, DCU support, and DeepSpeed finetuning

Added four new requirements files to define the project's Python dependencies: the main \requirements.txt\ pins \transformers\ (\>=4.32.0,\<4.38.0) and \transformers\_stream\_generator\ (0.0.4) alongside \accelerate\, \tiktoken\, \einops\, and \scipy\; \requirements\_web\_demo.txt\ introduces \gradio\ (\<3.42) and \mdtex2html\ for the web interface; \dcu-support/requirements.txt\ specifies dependencies for DCU hardware support including \streamlit\ (\>=1.24.0) and \sentencepiece\; and \recipes/finetune/deepspeed/requirements.txt\ adds \deepspeed\ and \peft\ for finetuning workflows.

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 44.8 → 43.1 (-1.7)
  • Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.

Lenses

  • Code Health 74.7 → 71.1 (-3.6)
  • Architecture 98.8 → 98.8 (0.0)
  • Maturity 47.2 → 47.2 (0.0)
  • Readiness 32.6 → 29.7 (-2.9)
  • Security 42.4 → 42.4 (0.0)
  • Performance 100.0 → 100.0 (0.0)

Resolved (6)

  • Duplicated block (10 lines × 3) (eval/evaluate_ceval.py)
  • Duplicated block (11–15 lines × 2) (eval/evaluate_ceval.py)
  • Duplicated block (18–22 lines × 2) (examples/function_call_finetune_examples.py)
  • Duplicated block (7 lines × 3) (eval/evaluate_ceval.py)
  • Edited copy of a member (19 corresponding lines) (examples/function_call_finetune_examples.py)
  • No ADRs found

New (20)

  • Edited copies of one member (3 members, 24+ corresponding lines) (eval/evaluate_ceval.py)
  • Edited copies of one member (3 members, 58+ corresponding lines) (eval/evaluate_ceval.py)
  • Edited copy of a member (12 corresponding lines) (eval/evaluate_chat_ceval.py)
  • Edited copy of a member (17 corresponding lines) (dcu-support/package/fastllm_pytools/llm.py)
  • Edited copy of a member (19 corresponding lines) (eval/evaluate_ceval.py)
  • Edited copy of a member (23 corresponding lines) (eval/evaluate_chat_mmlu.py)
  • Edited copy of a member (38 corresponding lines) (dcu-support/package/fastllm_pytools/llm.py)
  • Edited copy of a member (4 corresponding lines) (dcu-support/cli_demo.py)
  • Edited copy of a member (4 corresponding lines) (eval/evaluate_chat_ceval.py)
  • Edited copy of a member (41 corresponding lines) (finetune.py)
  • Edited copy of a member (50 corresponding lines) (eval/evaluate_chat_ceval.py)
  • Edited copy of a member (63 corresponding lines) (examples/vllm_wrapper.py)
  • Edited copy of a member (9 corresponding lines) (eval/evaluate_ceval.py)
  • Low coverage: cli_demo.py (cli_demo.py)
  • Low coverage: finetune.py (finetune.py)
  • Low coverage: openai_api.py (openai_api.py)
  • Low coverage: run_gptq.py (run_gptq.py)
  • Low coverage: utils.py (utils.py)
  • Low coverage: web_demo.py (web_demo.py)
  • No ADRs found (recipes/inference/tensorrt/docker/Dockerfile)

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

QwenLM/Qwen was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 16 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 2df8e8ac450fa185c421a08b0090ef81826caa6e — the exact code this score is about.
  • Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.83+preprod-a489942cf94b.