Skip to content
CAI
Software that uses CAICheck a score

deepspeedai/DeepSpeed

60.1

Adequate · 14 November 2026

122.9k

lines of production code

Python

with C, C++

6

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Features

Add Apple Silicon (MPS) support for DeepSpeed Adam optimizers

Users on Apple Silicon can now utilize DeepSpeed's Adam optimizer variants on the Metal Performance Shaders (MPS) backend. This change introduces builders for a Metal-based FusedAdam kernel (with a PyTorch foreach fallback) and a CPU-based Adam kernel for ZeRO-Offload scenarios, enabling optimized training workflows on macOS devices without requiring CUDA.

_op\builder/mps · high confidence

Add Biren SUPA accelerator support

Introduces a new \op\_builder/supa\ module that provides operator builders and Python wrappers for the Biren SUPA GPU accelerator. This includes builders for fused optimizers (Adam, LAMB, Lion), CPU optimizers (Adam, Lion, Adagrad), async I/O, and quantization, as well as a comprehensive inference module wrapping transformer kernels (workspace management, normalization, softmax, bias/GELU/ReLU ops, and residual additions). The implementation delegates to \torch.ops.deepspeed\ kernels provided by the \torch\_supa\_ext\ extension, with pure-PyTorch fallbacks for testing or environments where the compiled kernels are unavailable.

_op\builder/supa · high confidence

Add CPU Adagrad optimizer implementation

Introduces a new CPU-based Adagrad optimizer implementation in \csrc/adagrad/cpu\_adagrad.cpp\. This change adds support for running the Adagrad optimization algorithm on CPU hardware, featuring optimized paths for AVX2/AVX512 architectures and support for various data types including FP16, BF16, and FP32 for both parameters and state variables.

csrc/adagrad · high confidence

Add CPU-optimized Adam and Adagrad optimizers with SIMD and ZenFlow support

DeepSpeed now provides CPU-based implementations for the Adam and Adagrad optimizers, enabling training on machines without GPUs or offloading optimizer steps to the CPU. The CPU Adam implementation includes SIMD-accelerated kernels for AVX2, AVX-512, ARM NEON, and ARM SVE architectures, supports fp16/bf16 precision, and offers a ZenFlow mode that runs the optimizer in a separate native process to overlap computation with the Python training thread. The CPU Adagrad implementation similarly provides AVX2/AVX-512 vectorized updates. These components are exposed via C++ extensions and PyTorch bindings for integration into the DeepSpeed engine.

csrc/adam · high confidence

Add CPU-optimized Lion optimizer implementation

Users can now utilize the Lion optimizer on CPU devices through a new fused C++ extension. This change introduces \fused\_lion.cpp\, which implements the \multi\_tensor\_lion\ function to compute and apply gradient updates for parameters, exposing the functionality to Python via PyTorch's extension mechanism.

csrc/cpu/lion · high confidence

Add CPU-optimized optimizer kernels and CUDA utility headers

This change introduces new C++ header files in csrc/includes that provide CPU-optimized implementations for the Adam, Adagrad, and Lion optimizers, featuring SIMD acceleration (AVX, NEON, and ARM SVE) for training on CPU. It also adds utility headers for CUDA context management, timing (StopWatch, GPUTimer), data type conversions, and custom CUDA layers, alongside dequantization utilities to support mixed-precision and quantized training workflows.

csrc/includes · high confidence

Add Cambricon MLU operator builders for CPU-based optimizers

New builder classes are introduced in the \op\_builder/mlu\ module to support Cambricon MLU hardware, specifically enabling the build and loading of CPU-based optimizer implementations (CPU Adam, CPU Adagrad, and Fused Adam) via the \MLUOpBuilder\ base class. This addition allows the framework to recognize and compile these specific operator variants for MLU environments, while providing a placeholder for unimplemented operations.

_op\builder/mlu · high confidence

Add DeepSpeed4Science Evoformer attention kernels

This change introduces the C++ and CUDA implementation for the Evoformer attention mechanism within the DeepSpeed4Science module. It adds the PyTorch extension bindings in \attention.cpp\ and the core forward (\attention\_cu.cu\) and backward (\attention\_back.cu\) CUDA kernels, which utilize custom CUTLASS-based epilogues (including pipelined and logsumexp variants) and multi-architecture dispatch logic to support efficient attention computation with optional bias inputs.

csrc/deepspeed4science · high confidence

Add HPU op builder components

Added the op\_builder/hpu module, introducing builders for CPUAdam, FusedAdam, transformer inference, and FP quantization to support the HPU accelerator.

_op\builder/hpu · high confidence

Add KLXPU (Kunlunxin XPU) accelerator support

This change introduces a new op builder module for the KUNLUNXIN XPU accelerator, enabling DeepSpeed to build and load optimized operations on this hardware. The module provides builders for specific components including async I/O (with libaio dependency checks), pinned memory, and various CPU-based optimizers (Adam, Lion, Adagrad). It also includes a FusedAdam implementation that leverages KUNLUNXIN custom Torch ops when available, falling back to a pure-PyTorch implementation otherwise. Additionally, it defines stub builders for unsupported operations (such as FusedLamb, Transformer, and Quantizer) to ensure graceful handling of incompatible features on this backend.

_op\builder/klxpu · high confidence

Add XPU accelerator support for DeepSpeed op builder

Users can now build and load optimized C++ extensions for Intel XPU devices. This change introduces a new \op\_builder/xpu\ module containing a SYCL-based builder (\SYCLOpBuilder\) that compiles operations like CPU Adam, CPU Adagrad, Fused Adam, Async I/O, and Flash Attention for XPU. The builder ensures ABI compatibility by linking against the SYCL headers and libraries present in the Python environment rather than the system-wide oneAPI installation, and it handles JIT loading via the Intel \icpx\ compiler.

_op\builder/xpu · high confidence

Add fused LAMB optimizer kernel with ROCm support

This change introduces the fused LAMB (Layer-wise Adaptive Moments optimizer for Batch normalization) optimizer kernel for PyTorch. The implementation includes the C++ interface and CUDA/HIP kernel code, enabling optimized training performance. Notably, the kernel code includes conditional compilation logic to support both NVIDIA CUDA and AMD ROCm platforms, allowing the optimizer to run on AMD hardware in addition to NVIDIA GPUs.

csrc/lamb · high confidence

Added Metal FusedAdam kernel for Apple Silicon support

A new Metal shader implementation for the FusedAdam optimizer step has been added to the MPS backend, enabling hardware-accelerated training on Apple Silicon devices. The kernel mirrors the numerical behavior of the existing CUDA implementation by performing all math in fp32, ensuring consistent results for fp16 and bf16 parameters. It supports standard Adam and AdamW modes and includes specific instantiations for float and half-precision types, with conditional support for bfloat16 on Metal 3.1+ (macOS 14).

csrc/mps · high confidence

Added NVMe mount utilities and tensor casting helpers

This update introduces new utility scripts for managing NVMe storage on DGX-2 systems, specifically adding \dgx2\_mount\_nvme.sh\ and \dgx2\_umount\_nvme.sh\ to handle mounting and unmounting NVMe devices. Additionally, it adds C++ and Python bindings for tensor casting utilities (\tensor\_cast.cpp\, \tensor\_cast.h\, \py\_ds\_utils.cpp\) that allow casting 1D and multi-dimensional PyTorch tensors to byte tensors without data movement.

csrc/aio/utils, csrc/utils · high confidence

Added PR checklist agent skill

A new agent skill for pull request checklists has been introduced, implemented as a symbolic link pointing to the shared skills directory at ../../.agents/skills/pr-checklist.

.claude · high confidence

Added SDAA accelerator op builder support

Added the \op\_builder/sdaa\ module to enable building and loading optimized operators for the SDAA (Smart Data Accelerator Architecture) backend. This includes builders for CPU Adam and Fused Adam operations, allowing DeepSpeed to compile and utilize these specific kernel implementations when running on SDAA hardware.

_op\builder/sdaa · high confidence

Added optimized fused bias-add CUDA kernels and PyTorch bindings

This change introduces a new set of optimized CUDA kernels for fused bias-add operations in the spatial module, supporting three variants: standard bias addition, addition with a secondary tensor, and addition with two pairs of tensors and biases. These kernels are implemented in \opt\_bias\_add.cu\ and exposed to Python via new PyTorch bindings (\nhwc\_bias\_add\, \nhwc\_bias\_add\_add\, \nhwc\_bias\_add\_bias\_add\) in \pt\_binding.cpp\, enabling more efficient computation for half-precision tensors in deep learning models.

csrc/spatial · high confidence

CPU support for FusedAdam optimizer

Adds a new C++ implementation for the FusedAdam optimizer on CPU, exposing the \multi\_tensor\_adam\ function via PyTorch's C++ extension interface. This enables users to run the fused Adam optimization step on CPU hardware, mirroring the behavior available in the CUDA implementation, with support for different optimization modes (Adam vs AdamW) controlled by the \mode\ parameter.

csrc/cpu/adam · high confidence

GPUDirect Storage (GDS) support for DeepNVMe tensor swapping

This change introduces a new C++ and Python binding layer for GPUDirect Storage (GDS) within the DeepSpeed GDS component, enabling direct, high-performance swapping of optimizer tensors to and from NVMe storage devices. The implementation registers CUDA buffers with the cuFile driver to bypass the CPU, supporting both synchronous and asynchronous parallel read/write operations (pread/pwrite) with configurable queue depths and intra-operation parallelism. It also provides utilities for managing pinned device and CPU-locked tensors required for zero-copy I/O, effectively adding the underlying GDS capability that allows DeepNVMe to offload optimizer states directly to persistent storage.

csrc/gds · high confidence

Initial project configuration and contributor guidelines

The repository now includes foundational configuration files to standardize development workflows and governance. This adds \.clang-format\ and \.style.yapf\ for C++ and Python code formatting, \.flake8\ and \.pylintrc\ for Python linting, and a \.pre-commit-config.yaml\ to enforce these checks automatically before commits. It also introduces \AGENTS.md\ and \CLAUDE.md\ with specific rules for AI coding agents, \CONTRIBUTING.md\ detailing the submission and testing process, \GOVERNANCE.md\ outlining the Technical Steering Committee structure, \CODEOWNERS\ for PR review assignments, \CODE\_OF\_CONDUCT.md\ adopting the PyTorch Foundation standards, and \COMMITTERS.md\ listing the initial TSC members.

(repo-wide) · high confidence

Initial support for Huawei Ascend NPU operations

This change introduces the \op\_builder/npu\ module, enabling DeepSpeed to build and load custom C++ extensions for Huawei Ascend NPUs. It adds builders for key components including FusedAdam, CPU-based optimizers (Adam, Adagrad, Lion), asynchronous I/O, and inference-specific kernels (such as attention and layer norm). The implementation relies on the CANN toolkit and \torch\_npu\, requiring the \ASCEND\_HOME\_PATH\ environment variable to locate the necessary libraries and headers for compilation.

_op\builder/npu · high confidence

Introduce DataStates-LLM asynchronous checkpointing engine

DeepSpeed now supports an opt-in asynchronous checkpointing engine called DataStates-LLM. Users can enable this feature by adding a 'datastates\_ckpt' block to their configuration file and downloading the external DataStates-LLM library. This addition introduces new configuration classes and runtime modules to handle the integration, providing an alternative to the default synchronous checkpointing mechanism for potentially improved performance during checkpoint saves. Also in this area: CI sandbox image updated to Python 3.12; Optimize HybridEngine rollout memory by requesting last-position logits only; Improved numerical stability and performance for vocabulary-parallel operations; Improved checkpoint restoration accuracy and AutoSP validation; Introduce DualPipeV pipeline parallelism and batched point-to-point communication; and 59 more.

deepspeed · medium confidence

Introduce DeepCompile C++ backend for ZeRO-1, ZeRO-2, and ZeRO-3

This change adds the C++ implementation for the DeepCompile compiler integration, providing the core execution logic for ZeRO-1, ZeRO-2, and ZeRO-3 optimization stages. The new files in \csrc/compile\ define custom PyTorch operators (via \TORCH\_LIBRARY\_IMPL\) for parameter all-gathering, gradient reduction, and activation offloading/reloading. The implementation includes specific executors for each ZeRO stage (\z1.cpp\, \z2.cpp\, \z3.cpp\) that handle NCCL communication, symmetric memory usage, and CUDA stream management, along with initialization and utility functions to support these distributed training optimizations.

csrc/compile · high confidence

Introduce DeepSpeed Async I/O library with NVMe write warnings and CPU bounce buffers

This change adds the DeepSpeed Async I/O (AIO) C++/Python bindings in \csrc/aio/py\_lib\, exposing an \aio\_handle\ class and \aio\_read\/\aio\_write\ functions for swapping optimizer tensors to/from NVMe storage. The implementation includes a process-wide warning that fires once per process when writing to NVMe, alerting users that consumer SSDs may suffer reduced lifespan from heavy write traffic. It also introduces CPU bounce buffers for non-pinned tensors, ensuring data is copied to a pinned buffer before I/O, and supports file offsets for partial reads/writes. The library uses a multi-threaded worker model with configurable block size, queue depth, and intra-op parallelism, and integrates with DeepSpeed's pinned-tensor manager for efficient memory handling.

_csrc/aio/py\lib · high confidence

Introduce Modal-based GPU CI with diff-driven test selection and nightly triage

This change adds a new CI execution layer using Modal sandboxes for GPU workloads, including a PyTorch 2.9 image definition and a dedicated accelerate integration test runner. It introduces a diff-driven test selection mechanism (ci/check\_paths.py and ci/tests\_fetcher.py) that skips CI for documentation-only changes and narrows test scopes based on code impact, alongside a nightly full-suite run with automated failure classification and deduplicated issue reporting (ci/nightly\_classify.sh, ci/nightly\_report.sh). The change also includes comprehensive self-tests for the test selection logic and the Modal torch-latest runner, as well as AWS capacity fallback support for the Modal environment. Also in this area: 1 commit adding/updating tests in ci; CI gates now skip CPU/GPU workflows for version.txt changes.

ci · high confidence · unverified

Introduce common asynchronous I/O library for NVMe tensor swapping

Added a new shared asynchronous I/O (AIO) implementation in csrc/aio/common to handle swapping optimizer tensors to and from NVMe storage devices. This new codebase provides the core infrastructure for DeepSpeed's ZeRO-Offload and ZeRO-Infinity features, enabling efficient, non-blocking data movement between CPU memory and persistent storage to reduce GPU memory pressure during training.

csrc/aio/common · high confidence

Introduce native DeepSpeed host-memory pinning backend

Added a new native backend for page-locked CPU memory management in the pin\_memory module, implemented via new C++ sources (deepspeed\_pin\_tensor, page\_alloc) and exposed through a Python extension (py\_ds\_pin\_memory). This provides a dedicated mechanism for allocating, freeing, and checking page-locked tensors independent of other I/O libraries, supporting accelerators that require host memory pinning.

_csrc/pin\memory · high confidence

Introduces a unified accelerator abstraction layer

DeepSpeed now provides a standardized \DeepSpeedAccelerator\ interface that unifies support for diverse hardware backends. This change introduces dedicated implementations for CUDA, CPU, Apple Silicon (MPS), Habana HPU, Cambricon MLU, Ascend NPU, Intel XPU, Tecorigin SDAA, and Biren SUPA devices. The new \real\_accelerator\ module handles automatic device detection and allows users to explicitly override the target hardware via the \DS\_ACCELERATOR\ environment variable, ensuring consistent behavior across different accelerator types.

accelerator · high confidence

Introduction of new DeepSpeed CLI tools and launcher scripts

This change introduces a suite of new command-line interface scripts in the bin directory to enhance user interaction with DeepSpeed. The primary entry point is the \ds\ script (with \deepspeed\ and \deepspeed.pt\ symlinks), which launches the DeepSpeed runner using Python 3. New utility commands include \ds\_report\ for generating environment diagnostics, \ds\_bench\ for running communication benchmarks, \ds\_io\ for NVMe I/O operations, and \ds\_nvme\_tune\ for performance tuning of DeepNVMe storage. Additionally, \ds\_ssh\ is added to facilitate distributed training across hosts via SSH with configurable hostfiles, and Windows batch wrappers (\deepspeed.bat\, \ds\_report.bat\) are provided for cross-platform compatibility.

bin · high confidence

New CPU op builder module with FusedAdam, CPUAdam, and AsyncIO support

A new \op\_builder/cpu\ package has been introduced to manage the compilation of CPU-specific DeepSpeed operations. This module provides builders for FusedAdam and CPUAdam (enabling CPU training with mixed precision support), AsyncIO (leveraging libaio for non-blocking I/O), and communication primitives like CCLComm and ShareMemComm. It also includes a PinMemory builder and a fallback NotImplementedBuilder, allowing users to build and utilize these CPU-optimized components directly.

_op\builder/cpu · high confidence

New CPU shared-memory communication backend with multi-architecture vectorization

A new shared-memory (SHM) based communication backend for CPU inference has been added, introducing low-latency all-reduce operations for single-node scenarios. The implementation includes architecture-specific optimized kernels for x86\_64 (AVX-512), ARM64 (NEON), and RISC-V (RVV), enabling efficient vectorized reduction of BF16, FP16, and FP32 data types. The backend registers \deepspeed::inference\_all\_reduce\ as a native CPU operator, allowing users to perform collective communication directly on CPU tensors without falling back to other backends.

csrc/cpu/comm · high confidence

New CUDA kernel implementations for inference operations

The inference engine now includes a new set of optimized CUDA kernels for core transformer operations, including rotary positional embeddings, dequantization, GELU, LayerNorm, ReLU, RMSNorm, Softmax, and pointwise vector addition. These kernels, exposed via a new PyTorch C++ binding (pt\_binding.cpp), provide the low-level compute primitives required for the Hybrid Engine's inference path, supporting float, half, and bfloat16 data types.

csrc/transformer/inference/csrc · high confidence

New CUDA-based quantization and dequantization kernels

This change introduces a new set of CUDA kernels for quantization and dequantization operations, supporting both 4-bit and 8-bit precision with symmetric and asymmetric quantization types. The implementation includes optimized kernels for standard quantization, dequantization, and swizzled quantization (which reorganizes quantized groups to facilitate better communication in distributed settings). It also provides PyTorch bindings (pt\_binding.cpp) to expose these operations to Python, allowing users to perform quantization and dequantization directly on CUDA tensors. Additionally, a new kernel for quantized reduction is included, enabling efficient reduction operations on quantized data.

csrc/quantization · high confidence

New FP quantization implementation with selective dequantization support

The \csrc/fp\_quantizer\ module now includes a new C++/CUDA implementation for floating-point quantization. This change introduces support for quantizing tensors to various bit-widths (FP4, FP6, FP8, and FP12) with configurable mantissa and exponent bits, including optional stochastic rounding for improved accuracy. A key behavioral addition is the new \selective\_dequantize\ function, which allows dequantizing only specific groups of data identified by an index array, rather than the entire tensor. The module also exposes \get\_scales\ to retrieve the scaling factors used during quantization, enabling users to inspect or reuse these values for downstream operations.

_csrc/fp\quantizer · high confidence

New SDMA-accelerated ZeRO-3 example for AMD GPUs

The examples directory now includes a new \sdma\_allgather\ demo that showcases a hardware-accelerated AllGather path for ZeRO-3 on AMD MI300-series GPUs. By setting the \DS\_SDMA\_ALLGATHER=1\ environment variable, users can route collective traffic through the dedicated System DMA (SDMA) engines via the \mori\ library, bypassing the compute units to improve overlap with GEMM and attention workloads. The example provides scripts for both GPT-2-style and Qwen3-32B models, demonstrating approximately 10% step-time improvements while maintaining numerical accuracy compared to the standard RCCL/NCCL baseline.

examples · high confidence

New benchmarking tools for pinned memory and multimodal sequence parallelism

Added new benchmark scripts to the benchmarks directory: a pinned-memory host-to-device/device-to-host bandwidth test (benchmarks/pin\_memory/h2d\_d2h\_bench.py) that compares torch and native pinned-memory backends, and an AutoSP multimodal sequence-parallelism benchmark (benchmarks/autosp/bench\_multimodal\_sp.py) for ViT+LLM architectures (InternVL, Qwen2VL) to measure latency, throughput, and memory usage. A README was also added to point users to external communication and inference benchmark suites.

benchmarks · high confidence

New fused CUDA transformer kernels for DeepSpeed

This change introduces a new set of highly optimized, fused CUDA kernels for the transformer module, including implementations for attention softmax, GELU activation, layer normalization, dropout, and matrix transposition. These kernels are designed to improve training and inference performance by reducing memory bandwidth usage and kernel launch overhead through fusions like bias-add-GELU and fused bias-residual-layer-norm. The implementation includes C++ wrappers and Python bindings to integrate these custom CUDA operations into the DeepSpeed training pipeline, supporting both FP32 and FP16 (half-precision) data types.

csrc/transformer · high confidence

New inference context and CUDA layer abstractions for transformer models

This change introduces new header files in the inference includes directory that define the core runtime context and CUDA kernel launchers for transformer inference. The \InferenceContext\ class manages GPU workspace allocation, handling memory checks and cublas handle initialization, while the new \inference\_cublas\_wrappers.h\ provides a unified interface for GEMM operations that supports both NVIDIA CUDA and AMD ROCm backends (including backward compatibility for older Torch versions). Additionally, \inference\_cuda\_layers.h\ exposes template-based launchers for essential transformer operations such as attention softmax, bias-activation fusions (gelu, relu), layer normalization, residual connections, and rotary positional embeddings, enabling the underlying inference engine to execute these compute-intensive steps efficiently.

csrc/transformer/inference/includes · high confidence

New inference module interfaces and CUDA implementations for normalization layers

The DeepSpeed inference v2 module system now includes base interfaces and CUDA implementations for pre-normalization (LayerNorm and RMSNorm) and post-normalization layers. This adds \DSPreLNCUDAModule\ and \DSPreRMSCUDAModule\ for pre-norm operations, and \DSPostLNCUDAModule\ for post-norm, alongside their corresponding base classes (\DSPreNormBase\, \DSPostNormBase\) and registry structures. These components enable optimized, in-place normalization operations during inference, supporting both standard LayerNorm and RMSNorm variants with configurable epsilon and dtype constraints.

_deepspeed/inference/v2/modules/implementations/post\_norm, deepspeed/inference/v2/modules/implementations/pre\norm, deepspeed/inference/v2/modules/interfaces · high confidence

Architecture

Introduction of the op\_builder package for accelerator-aware custom op compilation

The \op\builder\ directory now provides a unified, abstracted interface for building DeepSpeed's custom C++/CUDA/ROCm extensions. This change introduces a new \OpBuilder\ base class and a registry system (\\\init\\_.py\, \all\_ops.py\) that dynamically discovers and instantiates builders (such as \CPUAdamBuilder\, \FusedAdamBuilder\, \AsyncIOBuilder\, and \FPQuantizerBuilder\) based on the current accelerator. This architecture allows the build system to automatically handle platform-specific requirements (e.g., ROCm vs. CUDA, CPU vs. GPU) and ensures that compatibility checks and compilation flags are applied correctly for each operator without hardcoding platform logic in individual modules.

_op\builder · high confidence

Behavioural changes

5 commits (0 fixes) modifying csrc/xpu/adam

A change to existing behaviour in csrc/xpu/adam — 5 commits, 3 files: “make xpu ops compatible with oneapi 2025.0 (\#6760)”; “Code cleaning: remove commented out debug prints and code (\#9036)”; “XPU use stock pytorch instead of Intel Extension for PyTorch (\#7877)”; and 2 more (refs \#6760, \#9036, \#7877, \#5702, \#4547).

csrc/xpu/adam · medium confidence · unverified

5 commits (2 fixes) modifying csrc/xpu/includes

A change to existing behaviour in csrc/xpu/includes — 5 commits (2 fixs), 6 files: “Update clang-format version from 16 to 18. (\#5839)”; “Fix Adam subgroup inconsistency (\#7982)”; “Code cleaning: remove commented out debug prints and code (\#9036)”; and 2 more (refs \#5839, \#7982, \#9036, \#8781, \#4547).

csrc/xpu/includes · medium confidence · unverified

Automated release workflow with version validation and patch bumping

The release process now includes automated checks to ensure the version number provided matches the repository's current version before building and uploading the package. A new \release.sh\ script validates the input, verifies the version against \version.txt\, builds the source distribution using \python -m build --sdist\, uploads it via twine, and creates a git tag. Additionally, a \bump\_patch\_version.py\ script is introduced to automatically increment the patch version after a release, and a \check\_release\_version.py\ script ensures consistency between the release version and the repository state.

release · high confidence

Introduce FP quantizer header infrastructure with AMD BF16 build fix

This change adds the \fp\_context.h\ and \fp\_quantize.h\ headers to the \csrc/fp\_quantizer/includes\ directory, establishing the core context management and quantization/dequantization API for the FP quantizer. A key behavioral adjustment is the conditional inclusion of \cuda\_bf16.h\: it is now wrapped in a \BF16\_AVAILABLE\ check and explicitly excluded on AMD platforms (where it would otherwise resolve to \hip/hip\_bfloat16.h\), preventing host-only compiler build errors by using a forward declaration instead. The headers also define macros for quantization bit-switching and declare the launch functions for quantization, dequantization, and selective dequantization.

_csrc/fp\quantizer/includes · high confidence

New pre-commit scripts to enforce code standards and accelerator abstraction

Added four new pre-commit check scripts: \check-torchcuda.py\ enforces the use of \deepspeed.accelerator.get\_accelerator()\ instead of direct \torch.cuda\ calls to support hardware abstraction; \check-torchdist.py\ requires \deepspeed.comm\ instead of \torch.distributed\; \check-extraindexurl.py\ prevents the use of \--extra-index-url\ in favor of \--index-url\; and \check-license.py\ ensures all source files contain the correct Microsoft DeepSpeed copyright and Apache 2.0 license headers. A companion script, \replace\_copyright.py\, is also added to help automate the replacement of copyright headers across Python, C/C++, and Bash files.

scripts · high confidence

Updated base images and added Python 3.11/3.12 build environments

The Docker build environment has been updated to use newer base images, specifically switching the main CUDA image to nvidia/cuda:12.2.2-devel-ubuntu20.04 and the PyTorch version to 1.13.0. Additionally, new Dockerfiles have been introduced in the gh-builder directory to support building with Python 3.11 and Python 3.12, providing users with updated language runtime options for development and testing.

docker · high confidence

Test coverage

1 commit adding/updating tests in tests/unit/v1/models; 10 commits adding/updating tests in tests/unit/v1/ops/triton\_ops; 11 commits adding/updating tests in tests/unit/v1; 11 commits adding/updating tests in tests/unit/v1/sequence\_parallelism; 12 commits adding/updating tests in tests/unit/v1/autotp; 48 commits adding/updating tests in tests/unit/v1/moe; 6 commits adding/updating tests in tests/unit/v1/accelerator; Added BingBertSquad model test suite; Added benchmark scripts for DeepSpeed4Science and Triton kernels; Added comprehensive test coverage for the Muon optimizer; Added debugging and validation tests for ZeRO-3 partial offload and memory behavior; Added initial PyTorch Lightning integration test; Added model sanity check test runner; Added performance benchmarks and accelerator initialization tests; Added tests for DeepCompile ZeRO-3 parameter management and graph break diagnostics; Added tests for DeepSpeed hybrid engine integration; Added tests for Gemma4 configuration validation; Added tests for Megatron GPT2 model integration; Added tests for PipelineModule checkpointing behavior; Added tests for the chunked causal-LM loss implementation; Added unit test for AIO handle destructor GIL release; Added unit tests for AutoEP communication, AutoTP partitioning, and Llama RoPE configuration; Added unit tests for AutoTP universal checkpoint metadata collection; Added unit tests for BF16, mixed-precision, and gradient clipping behaviors; Added unit tests for CPU Lion optimizer and configuration handling; Added unit tests for CPU and ZenFlow optimizer operations; Added unit tests for DeepCompile compilation and memory management; Added unit tests for DeepSpeed Hybrid Engine capabilities; Added unit tests for DeepSpeed compile subsystem; Added unit tests for DeepSpeed launcher argument parsing and multinode runners; Added unit tests for DeepSpeed linear optimization components; Added unit tests for DeepSpeed runtime components; Added unit tests for DeepSpeed utility functions; Added unit tests for DeepSpeed-FastGen v2 ragged inference components; Added unit tests for DeepSpeed4Science EvoformerAttention; Added unit tests for FP quantizer operations; Added unit tests for FlopsProfiler accuracy and stability; Added unit tests for INTx quantization and ZeRO-3 inference integration; Added unit tests for Inference V2 core modules; Added unit tests for Inference V2 kernel operations; Added unit tests for Inference V2 model implementation internals; Added unit tests for NHWC bias add spatial inference operations; Added unit tests for NVMe async I/O, GDS, and pinned memory management; Added unit tests for Ulysses SP and ALST components; Added unit tests for ZeRO runtime behaviors; Added unit tests for ZeRO state offloading, gradient reduction, and checkpoint alignment; Added unit tests for ZenFlow configuration validation and runtime behavior; Added unit tests for accelerator forward and backward passes; Added unit tests for accelerator graph capture, KLXPU, and MPS backends; Added unit tests for activation checkpointing and CPU offload; Added unit tests for autotuning components; Added unit tests for coalesced collective operations; Added unit tests for communication layer components; Added unit tests for distributed tensor partitioning utilities; Added unit tests for half-precision runtime behavior; Added unit tests for monitoring integrations; Added unit tests for native host memory pinning and accelerator integration; Added unit tests for op builder JIT and compatibility logic; Added unit tests for pipeline parallel runtime components; Added unit tests for sparse gradient handling and averaging; Added unit tests for the apply\_rotary\_pos\_emb sequence layer; Added unit tests for the quantizer inference operations; Added unit tests for transformer inference operators; Expanded unit test coverage for checkpointing, AutoTP, and AutoEP; New AIO benchmarking and validation test suite; New unit test infrastructure and fixtures; New unit tests for DeepSpeed inference capabilities; Standardize test infrastructure with pytest configuration and environment validation.

Dependencies

Standardize and pin Python dependencies across modular requirement files

The project now uses a set of modular requirement files (e.g., requirements.txt, requirements-dev.txt, requirements-readthedocs.txt) to manage dependencies more precisely. Key updates include pinning Pydantic to version 2.0.0 or higher across core and documentation builds, upgrading the development toolchain to use clang-format 18.1.3 and pre-commit 3.2.0, and constraining pytest to versions between 7.2.0 and 8.4.0. Additional changes include adding specific packages for autotuning (hjson, tabulate, xgboost), inference (lm-eval 0.3.0, safetensors), and Stable Diffusion support (diffusers \>=0.25.0, triton \>=2.1.0), while removing hard requirements for tensorboardX and megatron-lm.

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 61 → 60 (-1.4)
  • Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.

Lenses

  • Code Health 67 → 61 (-5.7)
  • Architecture 84 → 86 (+1.1)
  • Maturity 71 → 74 (+3.0)
  • Readiness 52 → 53 (+1.8)
  • Security 72 → 65 (-7.6)

Resolved (69)

  • Duplicated block (12 lines × 3) (deepspeed/ops/fp_quantizer/fp8_gemm_triton.py)
  • Duplicated block (15 lines × 2) (deepspeed/utils/comms_logging.py)
  • Duplicated block (16 lines × 2) (deepspeed/runtime/fp16/fused_optimizer.py)
  • Duplicated block (17 lines × 2) (deepspeed/ops/triton_ops/group_gemm_triton.py)
  • Duplicated block (17–18 lines × 3) (deepspeed/module_inject/containers/gptj.py)
  • Duplicated block (21 lines × 2) (op_builder/random_ltd.py)
  • Duplicated block (38 lines × 4) (deepspeed/ops/adagrad/cpu_adagrad.py)
  • Duplicated block (4–5 lines × 3) (deepspeed/module_inject/containers/clip.py)
  • Duplicated block (5 lines × 2) (csrc/aio/py_test/aio_bench_generate_param.py)
  • Duplicated block (5 lines × 2) (csrc/aio/py_test/aio_bench_perf_sweep.py)
  • Duplicated block (5 lines × 2) (csrc/aio/py_test/aio_bench_perf_sweep.py)
  • Duplicated block (5 lines × 2) (csrc/aio/py_test/ds_aio_args.py)
  • Duplicated block (5 lines × 2) (deepspeed/checkpoint/universal_checkpoint.py)
  • Duplicated block (5 lines × 2) (deepspeed/runtime/zero/stage_1_and_2.py)
  • Duplicated block (5 lines × 3) (deepspeed/ops/transformer/inference/triton/layer_norm.py)
  • Duplicated block (5–6 lines × 3) (deepspeed/runtime/zero/test.py)
  • Duplicated block (6 lines × 2) (deepspeed/runtime/data_pipeline/data_sampling/data_analyzer.py)
  • Duplicated block (6 lines × 2) (deepspeed/runtime/zero/stage_1_and_2.py)
  • Duplicated block (6 lines × 3) (deepspeed/ops/transformer/inference/triton/layer_norm.py)
  • Duplicated block (7 lines × 2) (deepspeed/ops/transformer/inference/triton/layer_norm.py)
  • …and 49 more

New (230)

  • AutoEPMoELayer.expert_storage_affine_maps (cognitive 20) (deepspeed/module_inject/auto_ep_layer.py)
  • DeepSpeedCPUAdam.step (cognitive 20) (deepspeed/ops/adam/cpu_adam.py)
  • DeepSpeedEngine._cast_module_mixed_precision (cyclomatic 18) (deepspeed/runtime/engine.py)
  • DeepSpeedEngine._load_zero_checkpoint (cognitive 17) (deepspeed/runtime/engine.py)
  • DeepSpeedZeroOptimizer._offload_compact_grad_partitions (cognitive 17) (deepspeed/runtime/zero/stage_1_and_2.py)
  • DeepSpeedZeroOptimizer.scaled_global_norm (cognitive 17) (deepspeed/runtime/zero/stage_1_and_2.py)
  • DeepSpeedZeroOptimizer_Stage3._rigid_load_state_dict (cognitive 16) (deepspeed/runtime/zero/stage3.py)
  • DeepSpeedZeroOptimizer_Stage3._setup_for_real_optimizer (cognitive 17) (deepspeed/runtime/zero/stage3.py)
  • DeepSpeedZeroOptimizer_Stage3._unpin_offload_buffers (cyclomatic 20) (deepspeed/runtime/zero/stage3.py)
  • DeepSpeedZeroOptimizer_Stage3.step (cognitive 24) (deepspeed/runtime/zero/stage3.py)
  • DeepSpeedZeroOptimizer_Stage3.step (cyclomatic 16) (deepspeed/runtime/zero/stage3.py)
  • Duplicated block (11 lines × 2) (deepspeed/ops/triton_ops/swiglu_triton.py)
  • Duplicated block (11 lines × 2) (deepspeed/runtime/fp16/fused_optimizer.py)
  • Duplicated block (12–13 lines × 2) (deepspeed/runtime/fp16/fused_optimizer.py)
  • Duplicated block (13 lines × 2) (ci/torch_latest.py)
  • Duplicated block (18 lines × 2) (deepspeed/ops/triton_ops/group_gemm_triton.py)
  • Duplicated block (21–37 lines × 4) (deepspeed/ops/adagrad/cpu_adagrad.py)
  • Duplicated block (24 lines × 2) (deepspeed/runtime/fp16/fused_optimizer.py)
  • Duplicated block (7 lines × 2) (deepspeed/checkpoint/affine_transfer_executor.py)
  • Duplicated block (7 lines × 2) (deepspeed/ops/transformer/inference/triton/layer_norm.py)
  • …and 210 more

Changes since last survey

  • 80 commits — 66 feature/other, 14 fixes

By area

  • deepspeed/runtime — 19 commits
  • tests/unit — 19 commits
  • .github/workflows — 7 commits
  • deepspeed/checkpoint — 7 commits
  • (root) — 4 commits
  • deepspeed/module_inject — 4 commits
  • ci/test_torch_latest.py — 3 commits
  • deepspeed/ops — 3 commits
  • deepspeed/comm — 2 commits
  • accelerator/cpu_accelerator.py — 1 commit
  • accelerator/cuda_accelerator.py — 1 commit
  • accelerator/klxpu_accelerator.py — 1 commit
  • accelerator/xpu_accelerator.py — 1 commit
  • ci/torch_latest.py — 1 commit
  • csrc/adam — 1 commit
  • csrc/aio — 1 commit
  • deepspeed/models — 1 commit
  • deepspeed/sequence — 1 commit
  • docs/code-docs — 1 commit
  • op_builder/builder.py — 1 commit

Notable commits

  • fix: DeepNVMe: fix silent I/O failures on misaligned buffers (#8985)
  • fix: Fix AutoEP storage-map transfer test for CPU tensor views (#9012)
  • fix: Fix bug in Adagrad optimizer initialization (#8976)
  • fix: Fix memory leak in engine and zero optimizer (#9034)
  • fix: Fix performance regression caused by comms logger and bug in comms_logger.enabled (#8909)
  • fix: Fix race between folded checkpoint load and metadata drop in AutoEP+AutoTP test (#8943)
  • fix: Fix routed-expert gradient clipping for AutoEP + AutoTP folding (#8823)
  • fix: Fix scalar parameter byte-view broadcasts (#8642)
  • fix: Fix torch-latest workflow unit failures (#8950)
  • fix: ZeRO-3 NVMe offload: fix "param already assigned swap buffer" in the parameter prefetch (#9014)
  • fix: [NPU] Fix the fused LAMB step and inference int4/int8 dequantization on Ascend (#9043)
  • fix: ci: pin torch<2.15 for MPS workflow until pytorch/pytorch#202529 is fixed (#8955)
  • fix: fix test_multiple_request_boundaries_before_step (#9028)
  • fix: fix(rollout): pass a pre-built 4-D mask to the static graph decode forward (#9004)
  • change: Add a v1 multi-rank CPU suite (manual dispatch first) (#8926)
  • change: Add an opt-in deterministic execution policy (#8917)
  • change: Add deepspeed.models.glm_moe_dsa: TileLang sparse attention for GLM-5.2 (#8659)
  • change: Add klxpu (KUNLUNXIN XPU) self-hosted CI for the v1 unit suite (#8923)
  • change: AutoEP + AutoTP folding: allow CPU optimizer offload under ZeRO-3 (#8829)
  • change: AutoEP activation tests: skip the Triton grouped-GEMM path when Triton cannot run (#8996)
  • …and 60 more

Architecture

  • Unchanged — 0 containers · 1 contexts · 0 edges

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

deepspeedai/DeepSpeed was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 14 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 663de9736f53e7f1e4deab75b7aa766694825b2a — the exact code this score is about.
  • Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-36d4cd83012f.