huggingface/diffusers
61.3
Adequate · 16 November 2026
807.1k
lines of production code
Python
primary language
5
measurements over time
What this system is
Features
Add AnyFlow video diffusion pipelines
Introduces two new pipelines for AnyFlow video generation: \AnyFlowPipeline\ for bidirectional text-to-video using flow-map-distilled checkpoints, and \AnyFlowFARPipeline\ for causal (chunk-wise autoregressive) text-to-video, image-to-video, and video-to-video generation. These pipelines support flexible inference step counts without retraining and include necessary output structures and lazy-loading configurations.
src/diffusers/pipelines/anyflow · high confidence
Add AnyText research example for multilingual visual text generation and editing
This update introduces the AnyText research project to the examples directory, providing a diffusion pipeline that generates or edits text within images. The addition includes the main pipeline implementation, a custom AnyTextControlNet model to handle text glyph and position conditioning, and a bundled OCR recognition module (based on MobileNetV1 and SVTR) for encoding stroke data. Users can now leverage this example to create images with specific text overlays or edit existing text in images, as demonstrated in the included README and usage code.
_examples/research\projects/anytext · high confidence
Add ChronoEdit pipeline for image-to-video generation
Users can now use the new ChronoEditPipeline to generate videos from images using the ChronoEdit model. This pipeline integrates a UMT5 text encoder, CLIP image encoder, AutoencoderKLWan VAE, and the ChronoEditTransformer3DModel, allowing for conditional video synthesis based on image and text prompts. The implementation includes support for Wan LoRA loading and handles video processing via the VideoProcessor utility.
src/diffusers/pipelines/chronoedit · high confidence
Add CogVideoX video generation pipelines
Introduces the CogVideoX pipeline suite for text-to-video, image-to-video, video-to-video, and controlled video generation. This adds \CogVideoXPipeline\, \CogVideoXImageToVideoPipeline\, \CogVideoXVideoToVideoPipeline\, and \CogVideoXFunControlPipeline\ to the library, enabling users to generate videos using the CogVideoX model architecture with support for T5 text encoding, LoRA loading, and PyTorch/XLA acceleration.
src/diffusers/pipelines/cogvideo · high confidence
Add CogView3Plus text-to-image pipeline
Users can now generate images using the CogView3Plus model via the new \CogView3PlusPipeline\. This feature introduces the pipeline implementation, its specific output data class, and the necessary module exports, enabling integration with the existing Diffusers ecosystem for this model architecture.
src/diffusers/pipelines/cogview3 · high confidence
Add CogView4 Control training example for structural conditioning
This location introduces an example for training Control LoRAs and performing full fine-tuning on the CogView4 model to support structural controls like depth maps and poses. The provided \train\_control\_lora\_cogview4.py\ and \train\_control\_cogview4.py\ scripts allow users to condition the model by expanding input features to include control latents, with support for DeepSpeed optimization and validation logging via WandB or TensorBoard.
examples/cogview4-control · high confidence
Add CogView4 and CogView4 Control pipelines
Introduces the \CogView4Pipeline\ for text-to-image generation and the \CogView4ControlPipeline\ for control-conditioned generation. The implementation includes the \CogView4PipelineOutput\ dataclass, lazy-loading infrastructure in \\_\init\\_.py\, and support for LoRA via \CogView4LoraLoaderMixin\. The control pipeline extends the base pipeline to accept a \control\_image\ argument, enabling guided image synthesis.
src/diffusers/pipelines/cogview4 · high confidence
Add ColossalAI DreamBooth training example
Introduces a new example in \examples/research\_projects/colossalai\ that demonstrates how to train Stable Diffusion models using the ColossalAI framework. The entry includes the \train\_dreambooth\_colossalai.py\ script, which leverages ColossalAI's Gemini memory manager to optimize GPU/CPU memory usage during training, along with supporting files for dependencies (\requirement.txt\), inference (\inference.py\), and documentation (\README.md\).
_examples/research\projects/colossalai · high confidence
Add Control-LoRA inference example for SDXL
A new example script and documentation have been added to the research projects directory, demonstrating how to use Control-LoRA with Stable Diffusion XL. The example shows how to load Control-LoRA adapters into a ControlNet model to enable image generation conditioned on reference images (such as canny edges) using Stability AI's Control-LoRA weights.
_examples/research\_projects/control\lora · high confidence
Add Cosmos 3 example with multi-modal and multi-GPU inference support
The \examples/cosmos3\ directory now provides a runnable smoke-test CLI (\inference\_cosmos3.py\) and multi-GPU helpers (\cosmos\_parallel.py\) for the Cosmos3 model. Users can generate text-to-image, text-to-video, image-to-video, video-to-video, and text-to-video-with-sound outputs. The example also supports action generation modes (forward/inverse dynamics, policy) for robotics and autonomous vehicle domains. It includes built-in support for multi-GPU inference via context parallelism (sharding the sequence) and tensor parallelism (sharding model weights), allowing large checkpoints to run across multiple GPUs.
examples/cosmos3 · high confidence
Add Custom Diffusion training example
Introduces a new example for Custom Diffusion, a method to customize text-to-image models like Stable Diffusion using just a few images of a subject. The \examples/custom\_diffusion\ directory now includes \train\_custom\_diffusion.py\ for training, \retrieve.py\ for fetching real regularization images via CLIP retrieval, and \test\_custom\_diffusion.py\ containing pytest-based integration tests for the training script and checkpointing logic.
_examples/research\_projects/autoencoder\_rae, examples/research\_projects/multi\_subject\dreambooth · high confidence
Add DiT image generation pipeline
Users can now generate images using the DiT (Diffusion Transformer) architecture via the new \DiTPipeline\. This pipeline replaces the traditional UNet backbone with a transformer model (\DiTTransformer2DModel\) and accepts ImageNet class labels to guide generation, offering an alternative to text-conditioned diffusion models.
src/diffusers/pipelines/bria, src/diffusers/pipelines/dit · high confidence
Add DiffusionGemma pipeline for block-diffusion text generation
Introduces the DiffusionGemmaPipeline, enabling text generation via block-diffusion using the DiffusionGemmaForBlockDiffusion model. The pipeline supports both single and batched prompts (via \prompt\ or \messages\ arguments), optional image conditioning, and configurable generation parameters such as \gen\_length\, \num\_inference\_steps\, and early stopping based on EOS tokens. It relies on schedulers like BlockRefinementScheduler, DiscreteDDIMScheduler, or EntropyBoundScheduler to manage the denoising and refinement of token canvases, and returns outputs via the new DiffusionGemmaPipelineOutput dataclass containing generated sequences and optional decoded text.
_src/diffusers/pipelines/diffusion\gemma · high confidence
Add DreamBooth training scripts with scheduled Pseudo-Huber loss
New example scripts are available in the \research\_projects/scheduled\_huber\_loss\_training/dreambooth\ directory for training DreamBooth models using a scheduled Pseudo-Huber loss objective. The release includes \train\_dreambooth.py\ for standard Stable Diffusion training, \train\_dreambooth\_lora.py\ for LoRA-based adaptation, and \train\_dreambooth\_lora\_sdxl.py\ for SDXL LoRA training, providing users with research-oriented training configurations for these specific model architectures.
_examples/research\_projects/scheduled\_huber\_loss\training/dreambooth · high confidence
Add DreamLite text-to-image and image-edit pipelines
Introduces the DreamLite pipeline module, providing \DreamLitePipeline\ for standard text-to-image and instruction-based image editing, and \DreamLiteMobilePipeline\ for fast, few-step (default 4 steps) inference without classifier-free guidance. Both pipelines utilize a Qwen3VL multimodal encoder, a tiny VAE, and a specialized DreamLite UNet, and are exposed via the \diffusers\ package for immediate use.
src/diffusers/pipelines/dreamlite · high confidence
Add EasyAnimateV5.1 video generation pipelines
Introduces three new pipelines for EasyAnimateV5.1 video generation: \EasyAnimatePipeline\ for text-to-video, \EasyAnimateControlPipeline\ for control-to-video (using pose or other control signals), and \EasyAnimateInpaintPipeline\ for image-to-video inpainting. These pipelines support models like \alibaba-pai/EasyAnimateV5.1-12b-zh\ and include specific utilities for handling video latents, masks, and control inputs, enabling users to generate high-quality videos with text prompts, control conditions, or inpainting capabilities.
src/diffusers/pipelines/easyanimate · high confidence
Add ErnieImagePipeline for text-to-image generation
Introduces the new ErnieImagePipeline, enabling text-to-image generation using the ErnieImageTransformer2DModel, a Flux2-style VAE, and a Mistral3 text encoder. The pipeline supports optional prompt enhancement via a PE (Prompt Enhancement) model and includes LoRA loading capabilities through ErnieImageLoraLoaderMixin. It requires transformers version 5.0.0 or higher.
_src/diffusers/pipelines/ernie\image · high confidence
Add Flux image generation examples for PyTorch/XLA on TPUs
New example scripts have been added to the PyTorch/XLA inference research project to demonstrate image generation using the FLUX model on TPU devices. The \flux\_inference.py\ script provides a single-chip inference example optimized for Trillium (v6) TPUs using custom flash attention block sizes, while \flux\_inference\_spmd.py\ introduces a Single-Program Multiple-Data (SPMD) approach that shards the transformer weights across multiple chips, enabling inference on configurations like v5e-8 where the model does not fit on a single device. Both scripts include setup instructions, performance benchmarks, and support for profiling and caching.
_examples/research\_projects/pytorch\xla/inference · high confidence
Add GGUF quantization support for diffusion models
Users can now load and run models quantized in the GGUF format, which significantly reduces memory usage during inference. This change introduces the \GGUFQuantizer\ class and associated utilities to handle GGUF-specific weight loading, dequantization, and optional CUDA kernel acceleration. The implementation supports various quantization types (Q4\_0, Q8\_0, etc.) and integrates with existing Diffusers features like \modules\_to\_not\_convert\ and \keep\_in\_fp32\_modules\ to allow selective precision control.
src/diffusers/quantizers/gguf · high confidence
Add GLIGEN training example and demo
Added a new research example in \examples/research\_projects/gligen\ that provides scripts to train the GLIGEN (Open-Set Grounded Text-to-Image Generation) model on the COCO dataset. This includes \train\_gligen\_text.py\ for distributed training via Accelerate, \make\_datasets.py\ for preparing grounding data using RAM, Grounding DINO, and BLIP2, a \COCODataset\ class for data loading, a \demo.ipynb\ notebook for inference visualization, and a \README.md\ with setup and usage instructions.
_examples/research\projects/gligen · high confidence
Add GeoDiff molecule conformation research notebook
Added an experimental research notebook in the examples/research\_projects/geodiff directory that demonstrates how to generate stable 3D structures of molecules using the GeoDiff model and Diffusers. The notebook includes setup instructions for Colab, installation of dependencies like PyTorch and Conda, and code to run pretrained models on GEOM datasets to produce physically accurate molecular conformations from 2D graph inputs.
_examples/research\projects/geodiff · high confidence
Add GlmImagePipeline for text-to-image generation
Introduces the new GlmImagePipeline in the diffusers library, enabling text-to-image generation using the GLM-Image model architecture. This pipeline integrates an autoregressive vision-language encoder (GlmImageForConditionalGeneration) for token generation with a diffusion transformer (GlmImageTransformer2DModel) for image decoding, requiring transformers version 4.57.4 or later. The implementation includes the main pipeline class, output data structures, and lazy-loading support for optional dependencies like Torch and Transformers.
_src/diffusers/pipelines/glm\image · high confidence
Add HiDream Image generation pipeline
Introduces the HiDreamImagePipeline, enabling users to generate images using the HiDream-ai architecture. This new pipeline integrates multiple text encoders (CLIP, T5, Llama) and a transformer-based diffusion model, supporting features like LoRA loading and CPU offloading for memory efficiency.
_src/diffusers/pipelines/hidream\image · high confidence
Add Hunyuan-DiT ControlNet inference pipeline
Introduces the \HunyuanDiTControlNetPipeline\ and its module exports, enabling users to perform image generation using the Tencent Hunyuan-DiT model with ControlNet conditioning. The pipeline supports both English and Chinese prompts, utilizes \T5Tokenizer\ and \T5EncoderModel\ for text encoding, and handles standard image aspect ratios and shapes for 1024x1024 and other common resolutions.
_src/diffusers/pipelines/controlnet\hunyuandit · high confidence
Add HunyuanDiT pipeline for bilingual text-to-image generation
Introduces the HunyuanDiTPipeline, enabling image generation from English and Chinese prompts using the Tencent Hunyuan DiT model. The pipeline integrates a bilingual CLIP text encoder and an mT5 encoder (via T5Tokenizer) for robust text understanding, supports standard aspect ratios (1:1, 4:3, 3:4, 16:9, 9:16) with automatic shape mapping, and includes noise rescaling to improve image quality and reduce overexposure.
src/diffusers/pipelines/hunyuandit · high confidence
Add HunyuanImage and HunyuanImageRefiner pipelines
Users can now generate images using the Hunyuan-Image 2.1 model via two new pipelines: \HunyuanImagePipeline\ for text-to-image generation and \HunyuanImageRefinerPipeline\ for image refinement. These pipelines integrate the HunyuanImageTransformer2DModel, Qwen2.5-VL text encoders, and FlowMatchEulerDiscreteScheduler, supporting features like custom timesteps and optional guidance components.
_src/diffusers/pipelines/hunyuan\image · high confidence
Add HunyuanVideo1.5 text-to-video and image-to-video pipelines
Introduces the \HunyuanVideo15Pipeline\ for text-to-video generation and the \HunyuanVideo15ImageToVideoPipeline\ for image-to-video generation. The implementation includes a dedicated \HunyuanVideo15ImageProcessor\ that handles aspect-ratio-aware cropping and resizing, and a specific \HunyuanVideo15PipelineOutput\ class to structure the generated video frames.
_src/diffusers/pipelines/hunyuan\_video1\5 · high confidence
Add Intel optimization example for Stable Diffusion inference
Added a new example script and documentation in the Intel optimization research project directory that demonstrates accelerating Stable Diffusion inference on Intel platforms using Intel Extension for PyTorch. The example enables Bfloat16 support for the UNet, VAE, and text encoder components to improve performance on Intel Xeon processors, including options for DPMSolverMultistepScheduler and specific environment variable configurations for OpenMP and memory allocation.
_examples/research\_projects/intel\opts · high confidence
Add Kandinsky 2.1 text-to-image pipelines
Introduces the Kandinsky 2.1 pipeline suite, enabling text-to-image, image-to-image, and inpainting generation. The update adds \KandinskyPipeline\ for standard text-to-image, \KandinskyImg2ImgPipeline\ for image-to-image, \KandinskyInpaintPipeline\ for inpainting, and \KandinskyCombinedPipeline\ for unified access. It also includes \KandinskyPriorPipeline\ for generating image embeddings from text prompts and a \MultilingualCLIP\ text encoder, all supporting CPU offloading and lazy imports.
src/diffusers/pipelines/kandinsky, src/diffusers/pipelines/kolors · high confidence
Add Kandinsky 2.2 text-to-image fine-tuning examples
New training scripts and documentation are provided for fine-tuning the Kandinsky 2.2 prior and decoder models. The \examples/kandinsky2\_2/text\_to\_image/\ directory now includes \train\_text\_to\_image\_prior.py\ and \train\_text\_to\_image\_decoder.py\ for full model fine-tuning, as well as \train\_text\_to\_image\_lora\_prior.py\ and \train\_text\_to\_image\_lora\_decoder.py\ for parameter-efficient LoRA fine-tuning. The accompanying README details how to run these scripts using Accelerate, including examples for the Naruto dataset, multi-GPU training, and inference with the resulting models.
_examples/kandinsky2\2 · high confidence
Add Krea 2 modular pipeline support
Introduces a new modular pipeline implementation for the Krea 2 text-to-image model, including \Krea2ModularPipeline\ and \Krea2TurboModularPipeline\. This adds support for the standard Krea 2 checkpoint (using symmetric classifier-free guidance) and the distilled Krea 2 Turbo checkpoint (guidance-free). The implementation includes modular blocks for text encoding via Qwen3-VL, core denoising loops, and decoding through the Qwen-Image VAE, enabling users to generate images using the Krea 2 architecture within the modular pipeline framework.
_src/diffusers/modular\pipelines/krea2 · high confidence
Add Krea 2 text-to-image pipeline
Users can now generate images using the Krea 2 (K2) model via the new \Krea2Pipeline\. This pipeline integrates a Qwen3-VL text encoder, a Krea 2 Transformer, and a Qwen-Image VAE, supporting both base (midtrain) and few-step distilled (TDM) checkpoints with configurable inference steps and guidance scales.
src/diffusers/pipelines/krea2 · high confidence
Add LEdits++ pipelines for Stable Diffusion and Stable Diffusion XL
Introduces two new image-editing pipelines, \LEditsPPPipelineStableDiffusion\ and \LEditsPPPipelineStableDiffusionXL\, which implement the LEdits++ algorithm for guided image editing. These pipelines allow users to edit images using specific prompts and guidance parameters, supporting features like inversion, tiling, and slicing. The release includes the pipeline implementations, custom attention and smoothing utilities, and dedicated output data classes for both the diffusion and inversion steps.
_src/diffusers/pipelines/ledits\pp · high confidence
Add LLaDA2 discrete diffusion training and sampling examples
New example scripts have been added to the \examples/discrete\_diffusion\ directory to support the LLaDA2 model, which uses block-wise iterative refinement for text generation. Users can now train causal language models (such as Qwen, Llama, or Mistral) using a confidence-aware loss via \train\_llada2.py\, or perform inference with \sample\_llada2.py\ using the \LLaDA2Pipeline\ and \BlockRefinementScheduler\. The training script supports custom datasets or dummy data, while the sampling script offers options for memory offloading, various sampling methods, and post-mask editing.
_examples/discrete\diffusion · high confidence
Add LLaDA2 pipeline for discrete diffusion text generation
Users can now generate text using the LLaDA2 pipeline, which implements discrete diffusion via block-wise iterative refinement. This new capability allows users to instantiate \LLaDA2Pipeline\ with a causal language model, a \BlockRefinementScheduler\, and a tokenizer to produce text outputs by refining masked token sequences in blocks.
src/diffusers/pipelines/llada2 · high confidence
Add LTX Video modular pipeline
Introduces a new modular pipeline for LTX Video, enabling users to compose and customize the generation process using modular blocks. This includes the \LTXModularPipeline\ class, which serves as the main entry point, and a set of reusable components such as \LTXTextEncoderStep\, \LTXVaeDecoderStep\, and \LTXDenoiseStep\. The implementation also provides \LTXBlocks\ and \LTXAutoBlocks\ to define the default sequence of operations for text-to-video and image-to-video generation, allowing for greater flexibility in how the model is used.
_src/diffusers/modular\pipelines/ltx · high confidence
Add Latent Consistency Distillation training examples for Stable Diffusion and SDXL
This change introduces a new \examples/consistency\_distillation\ directory containing scripts and documentation for distilling Stable Diffusion and SDXL models using Latent Consistency Models (LCM). It includes \train\_lcm\_distill\_sd\_wds.py\ and \train\_lcm\_distill\_sdxl\_wds.py\ for full model distillation using WebDataset, as well as \train\_lcm\_distill\_lora\_sd\_wds.py\ and \train\_lcm\_distill\_lora\_sdxl.py\ for training LCM-LoRA adapters. The addition is accompanied by \README.md\ and \README\_sdxl.md\ with usage instructions and a \test\_lcm\_lora.py\ file to verify the LCM-LoRA training functionality.
_examples/consistency\distillation · high confidence
Add Latent Consistency Models pipelines
Introduces the \LatentConsistencyModelPipeline\ for text-to-image generation and the \LatentConsistencyModelImg2ImgPipeline\ for image-to-image generation. These new pipelines are designed to work with Latent Consistency Models (LCM), enabling significantly faster inference with fewer steps (typically 1–8 steps) compared to standard diffusion models. Both pipelines support standard features such as LoRA weights, IP-Adapters, and textual inversions, and are exported via the \diffusers.pipelines.latent\_consistency\_models\ module.
_src/diffusers/pipelines/latent\_consistency\models · high confidence
Add Latent Perceptual Loss (LPL) example for Stable Diffusion XL
Added a new research example in \examples/research\_projects/lpl\ that implements Latent Perceptual Loss (LPL) for training Stable Diffusion XL models. This feature introduces a perceptual loss mechanism that operates in the latent space of the VAE to bridge the disconnect between the diffusion model and the autoencoder decoder, aiming to improve image quality, fine detail preservation, and structural consistency. The example includes a training script (\train\_sdxl\_lpl.py\) and a loss implementation (\lpl\_loss.py\) that allow users to enable LPL via command-line arguments (e.g., \--use\_lpl\), with configurable options for loss type (MSE/L1), normalization strategies, outlier removal, and feature extraction depth.
_examples/research\projects/lpl · high confidence
Add LoRA fine-tuning example for Cosmos Predict 2.5
This change introduces a complete example for fine-tuning the Cosmos Predict 2.5 model using LoRA. It includes a training script (\train\_cosmos\_predict25\_lora.py\) that supports LoRA and DoRA adapters, an evaluation script (\eval\_cosmos\_predict25\_lora.py\) for inference with trained adapters, and helper utilities to download and preprocess the GR1 dataset. The example also provides shell scripts for launching training and evaluation, along with LLM-based evaluation prompts for assessing video instruction following and physical commonsense.
examples/cosmos · high confidence
Add LoRA text encoder support to Stable Diffusion training example
The \train\_text\_to\_image\_lora.py\ script in the \examples/research\_projects/lora\ directory has been updated to support Low-Rank Adaptation (LoRA) for the text encoder, in addition to the UNet. This allows users to fine-tune both the image generation and text encoding components of Stable Diffusion using LoRA, resulting in more expressive model adaptations while keeping the original weights frozen. The accompanying README has been updated to reflect these changes, including new command-line arguments for configuring LoRA parameters specific to the text encoder (e.g., \--lora\_text\_encoder\_r\ and \--lora\_text\_encoder\_alpha\).
_examples/research\projects/lora · high confidence
Add LucyEditPipeline for video-to-video editing
Users can now use the new LucyEditPipeline to perform video-to-video editing by appending a condition video to the channel dimension. This pipeline, based on the Wan architecture, supports two-stage denoising via an optional second transformer and allows loading LoRA adapters for style or content adjustments.
src/diffusers/pipelines/lucy · high confidence
Add Lumina2 text-to-image pipeline
Introduces the Lumina2Pipeline for text-to-image generation, supporting the Lumina-Image-2.0 model. The pipeline integrates a Gemma2 text encoder, a Lumina2Transformer2DModel, and a FlowMatchEulerDiscreteScheduler, and includes built-in LoRA support via Lumina2LoraLoaderMixin for fine-tuning capabilities.
src/diffusers/pipelines/lumina2 · high confidence
Add LuminaPipeline for text-to-image generation
Users can now generate images from text prompts using the Lumina-T2I model via the new \LuminaPipeline\. This pipeline integrates the Gemma text encoder, the LuminaNextDiT2D transformer, and the FlowMatchEulerDiscrete scheduler, supporting features like CPU offloading and custom timestep/sigma schedules.
src/diffusers/pipelines/lumina · high confidence
Add MiniMax Music 3 modular pipeline
Users can now generate music using the MiniMax Music 3 model via a new modular pipeline. This implementation handles lyrics- and caption-conditioned generation by chaining an autoregressive Qwen3 language model (for semantic codes), a flow-matching transformer (for Flow-VAE latents), and a vocoder (for 44.1 kHz stereo waveforms). The pipeline supports chunked processing with overlapping windows to manage long audio durations up to six minutes.
_src/diffusers/modular\_pipelines/minimax\music3 · high confidence
Add MiniMax-H3 modular pipeline support
Adds the MiniMax-H3 modular pipeline implementation, including the \MiniMaxH3ModularPipeline\ class, \MiniMaxH3Blocks\, and reference classes for audio, image, video, and general references. This enables users to generate video and audio content using the MiniMax-H3 model through the modular pipeline architecture, supporting text-to-video, image-to-video, and reference-to-video workflows with specific handling for keyframes, references, and rotary position embeddings.
_src/diffusers/modular\_pipelines/minimax\_h3, src/diffusers/pipelines/deepfloyd\if · high confidence
Add NVIDIA ModelOpt quantization backend
Users can now quantize models using NVIDIA ModelOpt. This change introduces the \NVIDIAModelOptQuantizer\ class, which handles both post-training quantization (via calibration and compression) and the loading of pre-quantized models. It supports FP8 quantization, allows excluding specific modules (including convolutional layers) from quantization, and adjusts memory limits to account for quantization overhead.
src/diffusers/quantizers/modelopt · high confidence
Add Nunchaku Lite single-file quantization support
Users can now apply Nunchaku Lite quantization to diffusion models using the new \NunchakuLiteQuantizer\. This feature introduces support for SVDQ W4A4 and AWQ W4A16 compression schemes, automatically downloading and loading the required Nunchaku kernels from the Hugging Face Hub. The implementation enforces hardware requirements, restricting NVFP4 precision to Blackwell (compute capability 10+) or newer GPUs, and INT4 to Turing (compute capability 7.5+) or newer, while explicitly blocking unsupported Hopper (compute capability 9) devices.
src/diffusers/quantizers/nunchaku · high confidence
Add ONNX Runtime unconditional image generation example
Introduces a new training script and documentation for unconditional image generation using ONNX Runtime to accelerate DDPM UNet model training. The example includes a README with setup instructions and a Python script that leverages ONNX Runtime's FP16 optimizer and ORTModule for training, supporting features like EMA, mixed precision, and dataset integration via Hugging Face Datasets.
_examples/research\_projects/onnxruntime/unconditional\_image\generation · high confidence
Add ONNXRuntime-accelerated text-to-image fine-tuning example
A new experimental training script (\train\_text\_to\_image.py\) and documentation are added to the \examples/research\_projects/onnxruntime/text\_to\_image\ directory, enabling users to fine-tune Stable Diffusion models using ONNXRuntime for accelerated training. The example demonstrates how to configure the training environment with Accelerate, use the Naruto dataset for demonstration, and apply specific hyperparameters like gradient checkpointing and mixed precision.
_examples/research\_projects/onnxruntime/text\_to\image · high confidence
Add Ovis-Image text-to-image pipeline
Users can now generate images using the Ovis-Image pipeline, which leverages a Qwen3 text encoder, a custom OvisImageTransformer2DModel, and a FlowMatchEulerDiscreteScheduler. The new \OvisImagePipeline\ class and its associated output structure are available in the \diffusers\ library for text-to-image generation tasks.
_src/diffusers/pipelines/nucleusmoe\_image, src/diffusers/pipelines/ovis\image · high confidence
Add PixArt-Alpha ControlNet example pipeline and training scripts
This change introduces a new research example for PixArt-Alpha ControlNet, providing the model implementation (PixArtControlNetAdapterModel and PixArtControlNetTransformerModel), a dedicated inference pipeline (PixArtAlphaControlnetPipeline), and scripts for fine-tuning and running inference. Users can now condition PixArt-Alpha image generation on control signals (such as HED edges) using the provided training and inference utilities.
_examples/research\projects/pixart · high confidence
Add PromptDiffusion pipeline and ControlNet model
Introduces a new research example for PromptDiffusion, including the \PromptDiffusionPipeline\ for text-to-image generation using Stable Diffusion with ControlNet guidance, a custom \PromptDiffusionControlNetModel\ that extends the standard ControlNet to handle query-conditioned embeddings, and a conversion script to transform original PromptDiffusion checkpoints into the Diffusers format.
_examples/research\projects/promptdiffusion · high confidence
Add PyTorch/XLA text-to-image training example for Stable Diffusion
A new training script and documentation have been added to the PyTorch/XLA research examples, enabling users to fine-tune Stable Diffusion models on TPU devices. The \train\_text\_to\_image\_xla.py\ script demonstrates how to use PyTorch/XLA's GSPMD feature for distributed data parallelism across TPU v4 and v5p accelerators, including setup instructions for Google Cloud TPUs and usage of nightly PyTorch/XLA versions.
_examples/research\_projects/pytorch\xla/training · high confidence
Add Qwen-Image 2.1 pipeline
Users can now generate images using the Qwen-Image 2.1 model via the new \QwenImage21Pipeline\. This pipeline supports both text-to-image and image-conditioned generation, leveraging a Qwen3-VL model for joint text and image encoding, an \AutoencoderKLQwenImage21\ VAE, and a \QwenImage21Transformer2DModel\ for denoising. It includes support for LoRA adapters through \QwenImageLoraLoaderMixin\ and is compatible with Torch XLA.
src/diffusers/pipelines/qwenimage21 · high confidence
Add RealFill example for personalized text2image inpainting
Introduces the RealFill research example, providing scripts (\train\_realfill.py\, \infer.py\) and documentation to train and run personalized text2image inpainting models using Stable Diffusion. The training script supports distributed training via Accelerate, LoRA fine-tuning, and memory optimization techniques like gradient checkpointing, 8-bit optimizers, and xformers, allowing users to personalize inpainting models with just a few reference images.
_examples/research\projects/realfill · high confidence
Add Retrieval Augmented Diffusion (RDM) research example
Introduces a new research example in \examples/research\_projects/rdm\ that implements Retrieval Augmented Diffusion Models. This includes the \RDMPipeline\ class, which extends \DiffusionPipeline\ to incorporate image retrieval via a \Retriever\ component using FAISS and CLIP embeddings, alongside supporting modules like \retriever.py\ for index management and image preprocessing.
_examples/research\projects/rdm · high confidence
Add SDXL example for JAX/Flax on TPU
The \examples/research\_projects/sdxl\_flax\ directory now includes a new example demonstrating how to run Stable Diffusion XL using JAX and Flax on TPU v5e hardware. The entry provides \sdxl\_single.py\ for standard JIT-compiled inference and \sdxl\_single\_aot.py\ for Ahead-of-Time compilation to optimize performance. The included README notes that JAX/Flax support was removed from the main \diffusers\ library in v0.40.0, so this example requires \diffusers\<=0.39.x\.
_examples/research\_projects/sdxl\flax · high confidence
Add Sana Sprint training example
Added a new example in \examples/research\_projects/sana\ that demonstrates how to train the Sana Sprint diffuser model. This includes a Python training script (\train\_sana\_sprint\_diffusers.py\), a bash execution script, and documentation (\README.md\) explaining how to download the pre-trained teacher model and run training on specific dataset shards.
_examples/research\projects/sana · high confidence
Add Shap-E 3D generation pipeline
Introduces the Shap-E pipeline, enabling users to generate 3D assets from text prompts or input images. This change adds the \ShapEPipeline\ for text-to-3D and \ShapEImg2ImgPipeline\ for image-to-3D, along with supporting components including a differentiable projective camera model, a NeRF-based renderer, and lazy import structures to manage optional dependencies like PyTorch and Transformers.
_src/diffusers/pipelines/shap\e · high confidence
Add SkyReels V2 video generation pipelines
Introduces the SkyReels V2 pipeline module, providing five new DiffusionPipeline implementations: SkyReelsV2Pipeline (text-to-video), SkyReelsV2ImageToVideoPipeline (image-to-video), and three variants using diffusion forcing (SkyReelsV2DiffusionForcingPipeline, SkyReelsV2DiffusionForcingImageToVideoPipeline, and SkyReelsV2DiffusionForcingVideoToVideoPipeline) to support longer video generation. The module includes the SkyReelsV2PipelineOutput dataclass and lazy-loading exports for all pipeline classes.
_src/diffusers/pipelines/skyreels\v2 · high confidence
Add ValueGuidedRLPipeline for value-guided diffusion in RL
A new \ValueGuidedRLPipeline\ is now available in the \diffusers.experimental.rl\ module. This pipeline enables value-guided sampling for diffusion models trained on state sequences, allowing users to fine-tune trajectories based on reward signals. It integrates a \UNet1DModel\ for value estimation and another for denoising, working with OpenAI Gym-compatible environments (with pretrained models currently available for the Hopper task). Users can control the strength of value guidance via \n\_guide\_steps\ and \scale\ parameters, and the pipeline automatically selects the trajectory with the highest estimated value.
src/diffusers/experimental/rl · high confidence
Add VisualCloze pipeline for image generation with visual context
Introduces the VisualCloze pipeline, enabling image generation based on visual in-context examples. This new capability includes the \VisualClozePipeline\ for combined generation and upsampling, the \VisualClozeGenerationPipeline\ for core generation, and the \VisualClozeProcessor\ utility for handling image preprocessing, resizing, and mask generation specific to the visual cloze task.
src/diffusers/pipelines/visualcloze · high confidence
Add Wan-Animate-2 modular pipeline and video processor
Introduces a new modular pipeline for the Wan-Animate-2 character animation model, providing both standard and distilled inference paths. This includes a custom \WanAnimate2VideoProcessor\ that implements specific letterboxing logic to match the reference implementation's pixel placement, alongside the full pipeline architecture (encoders, denoise steps, and decoders) required to drive the model.
_src/diffusers/modular\_pipelines/wan\_animate\2 · high confidence
Add Wuerstchen text-to-image prior fine-tuning examples
New example scripts and documentation have been added to the Wuerstchen research project to support fine-tuning the Wuerstchen prior model. This includes \train\_text\_to\_image\_prior.py\ for standard prior training and \train\_text\_to\_image\_lora\_prior.py\ for Low-Rank Adaptation (LoRA) fine-tuning, along with a custom \EfficientNetEncoder\ module and a comprehensive README explaining setup, usage, and hyperparameters.
_examples/research\projects/wuerstchen · high confidence
Add bitsandbytes 4-bit and 8-bit quantization support
Users can now load models using bitsandbytes quantization (4-bit NF4/FP4 and 8-bit) via the new \BnB4BitDiffusersQuantizer\ and \BnB8BitDiffusersQuantizer\ classes. This feature replaces standard \nn.Linear\ layers with \bnb.nn.Linear4bit\ or \bnb.nn.Linear8bitLt\ modules during model loading, significantly reducing memory usage. The implementation includes validation for required dependencies (Accelerate \>= 0.26.0, bitsandbytes \>= 0.43.3) and supports GPU, XPU, and MPS devices.
src/diffusers/quantizers/bitsandbytes · high confidence
Add consistency training example script
Added a new community example script (\train\_cm\_ct\_unconditional.py\) and its documentation (\README.md\) in the \examples/research\_projects/consistency\_training\ directory. This script enables users to train consistency models from scratch using the consistency training algorithm, supporting both unconditional and class-conditional training on datasets like CIFAR-10.
_examples/research\_projects/consistency\training · high confidence
Add experimental Diffusion DPO training examples for SD, SDXL, and SDXL Turbo
The \examples/research\_projects/diffusion\_dpo\ directory now includes training scripts (\train\_diffusion\_dpo.py\ and \train\_diffusion\_dpo\_sdxl.py\) and documentation for performing Direct Preference Optimization (DPO) on Stable Diffusion models using LoRA. Users can now train LoRA adapters on SD 1.5, SDXL, and SDXL Turbo models using preference datasets (such as \kashif/pickascore\), with support for validation logging to WandB and TensorBoard. These scripts are experimental and intended for low-data regime research.
_examples/research\_projects/diffusion\dpo · high confidence
Add modular pipeline for HunyuanVideo 1.5
Introduces a new modular pipeline implementation for the HunyuanVideo 1.5 model, enabling text-to-video and image-to-video generation. This change adds the \HunyuanVideo15ModularPipeline\ class and a suite of modular blocks (encoders, denoisers, decoders) that handle specific stages such as text encoding with Qwen2.5-VL and ByT5, latent preparation, iterative denoising with classifier-free guidance, and VAE decoding. The pipeline is exposed via the \diffusers.modular\_pipelines.hunyuan\_video1\_5\ module for use with the modular pipeline framework.
_src/diffusers/modular\_pipelines/hunyuan\_video1\5 · high confidence
Add modular pipeline for Stable Diffusion 3
Introduces a new modular pipeline implementation for Stable Diffusion 3, exposing \StableDiffusion3ModularPipeline\ and \StableDiffusion3AutoBlocks\ via the \diffusers.modular\_pipelines.stable\_diffusion\_3\ module. This change adds the core pipeline class and a suite of modular blocks—including text encoding, denoising, and decoding steps—that allow users to construct and customize the Stable Diffusion 3 generation process.
_src/diffusers/modular\_pipelines/stable\_diffusion\_3, src/diffusers/pipelines/consistency\models · high confidence
Add modular pipeline support for FLUX.3 image generation
Introduces a new modular pipeline implementation for the FLUX.3 model, providing \Flux3ModularPipeline\ and \Flux3DistilledModularPipeline\ classes along with their corresponding auto-blocks (\Flux3AutoBlocks\, \Flux3DistilledAutoBlocks\). This addition includes specialized pipeline steps for text encoding via Qwen3-VL, image conditioning (including reference image preprocessing and VAE encoding), core denoising loops (supporting both standard and step-distilled variants), and latent decoding using the FLUX.2 VAE. The implementation also features a dedicated \Flux3ImageProcessor\ to handle image resizing and padding according to the BFL reference specifications.
_src/diffusers/modular\pipelines/flux3 · high confidence
Add scheduled Huber loss training scripts for Stable Diffusion and SDXL
New example scripts have been added to the \examples/research\_projects/scheduled\_huber\_loss\_training/text\_to\_image\ directory to support fine-tuning Stable Diffusion and Stable Diffusion XL models using a scheduled Huber loss function. The addition includes \train\_text\_to\_image.py\ and \train\_text\_to\_image\_sdxl.py\ for standard fine-tuning, as well as \train\_text\_to\_image\_lora.py\ and \train\_text\_to\_image\_lora\_sdxl.py\ for LoRA-based adaptation. These scripts provide users with research-grade training configurations that differ from the standard examples, specifically implementing the scheduled Huber loss objective for text-to-image generation tasks.
_examples/research\_projects/scheduled\_huber\_loss\_training/text\_to\image · high confidence
Add server example for concurrent image generation
A new example server has been added to \examples/server\ that demonstrates how to use Diffusers pipelines (specifically Stable Diffusion 3) as a concurrent, multithreaded inference engine. The server, built with FastAPI and Uvicorn, exposes a \/v1/images/generations\ endpoint for text-to-image generation, handling requests asynchronously to allow multiple users to generate images simultaneously while managing GPU resources efficiently.
examples/server · high confidence
Add textual inversion example with Intel CPU optimizations
Introduces a new training example for Textual Inversion in the \examples/research\_projects/intel\_opts/textual\_inversion\ directory, specifically optimized for Intel CPUs using Intel Extension for PyTorch (IPEX). The release includes \textual\_inversion\_bf16.py\ to leverage bfloat16 precision on supported Intel Xeon processors (Cooper Lake or Sapphire Rapids) for faster training, along with a \README.md\ detailing setup and execution for both single-node and multi-node distributed training scenarios.
_examples/research\_projects/intel\_opts/textual\inversion · high confidence
Community pipeline documentation and examples consolidated
The \examples/community\ directory now contains a comprehensive \README.md\ that catalogs all available community-contributed pipelines. This single entry introduces the full directory structure and documentation for these examples, including pipelines for CLIP guidance, differential diffusion, HD-painting, Marigold depth estimation, LLM-grounded diffusion, and various SDXL-specific features like long prompt weighting and mixture tiling. Users can now find a centralized index of community scripts with descriptions, code examples, and Colab links.
examples/community · high confidence
Diffusers 0.41.0.dev0: Modular pipelines, new model support, and quantization backends
This release introduces the Modular Diffusers architecture, allowing users to construct pipelines from interchangeable blocks (e.g., for Wan, Qwen, and Flux models) rather than relying solely on monolithic pipeline classes. It adds official support for several new models and pipelines, including Anima, MiniMax Music 3, and various Flux variants (Inpaint, Img2Img, ControlNet). The library also expands its quantization capabilities with new backends for AutoRound, GGUF, Quanto, and TorchAO, while updating core dependencies such as \huggingface-hub\ (\>=1.32.0) and \torch\ (\>=2.6). Additionally, it introduces a new \PipelineCallback\ system for advanced inference control (e.g., CFG cutoff) and deprecates direct access to config attributes in favor of accessing them via the \.config\ object.
src/diffusers · high confidence
Introduce Anima modular pipeline for text-to-image and image-to-image generation
Adds the \AnimaModularPipeline\ and its constituent modular blocks (text encoding, conditioning, denoising, and decoding) to the \diffusers\ library. This new pipeline enables users to generate images from text prompts or modify existing images using the Anima model architecture, supporting both standard text-to-image and image-to-image workflows through a modular, composable structure.
_src/diffusers/modular\_pipelines/anima, src/diffusers/modular\pipelines/ideogram4 · high confidence
Introduce AudioLDM2 pipeline for text-to-audio and text-to-speech generation
Adds the AudioLDM2Pipeline, enabling users to generate audio from text prompts or synthesize speech from text transcriptions. The implementation includes the AudioLDM2UNet2DConditionModel and AudioLDM2ProjectionModel, which integrate CLAP, T5/VITS, and GPT-2 encoders to process text conditions for the diffusion process. The pipeline supports both standard text-to-audio generation and text-to-speech (TTS) modes, requiring transformers version 4.27.0 or higher.
src/diffusers/pipelines/audioldm2 · high confidence
Introduce AuraFlow pipeline for text-to-image generation
Adds the AuraFlow pipeline, enabling users to generate images using the AuraFlow model architecture. This new feature includes support for the T5 text encoder, the AuraFlowTransformer2DModel, and the FlowMatchEulerDiscreteScheduler. The implementation provides LoRA loading capabilities via AuraFlowLoraLoaderMixin, supports custom timestep and sigma schedules, and includes compatibility with Torch XLA for accelerated inference on compatible hardware.
_src/diffusers/pipelines/aura\flow · high confidence
Introduce BriaFibo and BriaFiboEdit pipelines
Adds the \BriaFiboPipeline\ and \BriaFiboEditPipeline\ classes to the \diffusers\ library, enabling image generation and editing using the Bria Fibo model architecture. The \BriaFiboPipeline\ supports text-to-image generation with a \SmolLM3\ text encoder and \AutoencoderKLWan\ VAE, while the \BriaFiboEditPipeline\ extends this with multi-reference conditioning and image editing capabilities, including support for multiple reference images and mask inputs. Both pipelines include lazy loading support and integrate with the existing Diffusers infrastructure.
_src/diffusers/pipelines/bria\fibo · high confidence
Introduce Chroma pipeline family for text-to-image, img2img, and inpainting
Adds the \ChromaPipeline\ (text-to-image), \ChromaImg2ImgPipeline\ (image-to-image), and \ChromaInpaintPipeline\ (inpainting) to the \diffusers\ library, enabling generation with the \lodestones/Chroma1-HD\ model. These pipelines support features such as LoRA loading, single-file checkpoint loading, textual inversion, and IP-Adapter, and are exported via the \src/diffusers/pipelines/chroma\ module.
src/diffusers/pipelines/chroma · high confidence
Introduce ConsisID pipeline for identity-consistent video generation
Added the ConsisID pipeline, which enables generating videos that preserve the identity of a person shown in a reference image. The implementation includes the main pipeline class, utility functions for face detection and embedding extraction (using insightface, facexlib, and consisid\_eva\_clip), and a dedicated output class. Users can now import ConsisIDPipeline to perform identity-consistent video synthesis, requiring optional dependencies like OpenCV, PyTorch, and Transformers.
src/diffusers/pipelines/consisid · high confidence
Introduce ControlNet Union pipelines for Stable Diffusion XL
The \src/diffusers/pipelines/controlnet\ module now includes three new pipeline classes—\StableDiffusionXLControlNetUnionPipeline\, \StableDiffusionXLControlNetUnionInpaintPipeline\, and \StableDiffusionXLControlNetUnionImg2ImgPipeline\—alongside their standard SDXL counterparts. These new pipelines are designed to work with the ControlNet Union model, allowing users to apply multiple conditioning types (such as Canny, depth, and pose) simultaneously or interchangeably within a single unified pipeline architecture.
src/diffusers/pipelines/controlnet · high confidence
Introduce Custom Diffusion training example with real-image regularization
Adds a new \custom\_diffusion\ example that demonstrates how to customize text-to-image models (like Stable Diffusion) using only a few images of a subject. The training script (\train\_custom\_diffusion.py\) supports single and multi-concept training, integrates with Weights & Biases for experiment tracking, and allows pushing results to the Hugging Face Hub. To prevent overfitting, the example includes a \retrieve.py\ utility that downloads real regularization images from LAION using \clip-retrieval\, alongside a pytest-based test suite (\test\_custom\_diffusion.py\) to validate training and checkpointing behavior.
_examples/custom\diffusion · high confidence
Introduce DDIMPipeline for image generation
Adds the DDIMPipeline class to the diffusers library, enabling users to generate images using the DDIM (Denoising Diffusion Implicit Models) algorithm. The pipeline accepts a UNet2DModel and a DDIMScheduler, supports configurable batch sizes, inference steps, and the eta parameter to interpolate between DDIM and DDPM sampling behaviors, and returns outputs in PIL, NumPy, or PyTorch tensor formats. It includes validation for the eta parameter, handles device execution (including PyTorch/XLA support), and manages model offloading.
src/diffusers/pipelines/ddim · high confidence
Introduce DDPM image generation pipeline with lazy loading and XLA support
Adds the \DDPMPipeline\ class for unconditional image generation, allowing users to load a pre-trained DDPM model and generate images via \pipe()\. The pipeline supports configurable batch size, inference steps, and output formats (PIL, NumPy, or PyTorch tensors). It includes lazy module loading for faster imports and explicit support for PyTorch/XLA devices via \xm.mark\_step()\ calls during the denoising loop.
src/diffusers/pipelines/ddpm · high confidence
Introduce ErnieImage modular pipeline for text-to-image generation
Adds a new modular pipeline implementation for the ErnieImage model, enabling text-to-image generation with an optional prompt enhancer (Ministral3ForCausalLM) and support for LoRA loading. The pipeline is composed of distinct processing steps: prompt enhancement, text encoding (Mistral3Model), denoising (ErnieImageTransformer2DModel with classifier-free guidance), and decoding (AutoencoderKLFlux2). It requires transformers version 5.0.0 or higher for the prompt enhancer components.
_src/diffusers/modular\_pipelines/ernie\image · high confidence
Introduce FLUX.2 pipeline family (base, Klein, Klein Inpaint, Klein KV)
Adds the \src/diffusers/pipelines/flux2\ module with four new pipelines—\Flux2Pipeline\, \Flux2KleinPipeline\, \Flux2KleinInpaintPipeline\, and \Flux2KleinKVPipeline\—alongside supporting components (\Flux2ImageProcessor\, \Flux2PipelineOutput\, and system messages). Users can now generate images with the FLUX.2 model variants, including inpainting with optional reference conditioning and faster inference via KV-cached reference images in the Klein KV pipeline.
src/diffusers/pipelines/flux2 · high confidence
Introduce Flux pipeline family
Adds the Flux pipeline module, providing a complete set of generation pipelines for the Flux model architecture. This includes the base text-to-image pipeline, along with specialized variants for image-to-image, inpainting, and controllable generation (ControlNet and Control pipelines). The module also exports supporting components such as the Flux model output classes and the Redux image encoder used for prior-based workflows.
src/diffusers/pipelines/flux · high confidence
Introduce Helios modular video generation pipeline
Adds the Helios modular pipeline to Diffusers, providing a new way to generate videos using the Helios model. This includes the core pipeline class (HeliosModularPipeline) and a set of modular blocks for text encoding, image/video encoding, chunk-based denoising (with support for standard, pyramid, and distilled pyramid modes), and decoding. The implementation supports text-to-video, image-to-video, and video-to-video workflows, leveraging the new HeliosTransformer3DModel and HeliosScheduler components.
_src/diffusers/modular\pipelines/helios · high confidence
Introduce HunyuanVideo pipeline family
Adds the \src/diffusers/pipelines/hunyuan\_video\ module, exposing four new pipelines for video generation: \HunyuanVideoPipeline\ (text-to-video), \HunyuanVideoImageToVideoPipeline\ (image-to-video), \HunyuanSkyreelsImageToVideoPipeline\ (SkyReels image-to-video), and \HunyuanVideoFramepackPipeline\ (image-to-video with framepack support). These pipelines integrate HunyuanVideo-specific components, including the \HunyuanVideoTransformer3DModel\, \AutoencoderKLHunyuanVideo\, and \FlowMatchEulerDiscreteScheduler\, and provide dedicated output classes (\HunyuanVideoPipelineOutput\, \HunyuanVideoFramepackPipelineOutput\) for returning generated video frames.
_src/diffusers/pipelines/hunyuan\video · high confidence
Introduce Ideogram4 text-to-image pipeline
Adds the \Ideogram4Pipeline\ component, enabling users to generate images using the Ideogram4 flow-matching model. The pipeline integrates a \FlowMatchEulerDiscreteScheduler\ with a resolution-aware logit-normal schedule, supports asymmetric classifier-free guidance via separate conditional and unconditional transformers, and includes optional structured prompt upsampling through the \Ideogram4PromptEnhancerHead\ to convert free-form prompts into the model's native JSON caption schema. It also exposes LoRA loading capabilities via \Ideogram4LoraLoaderMixin\ and defines the \Ideogram4PipelineOutput\ data class for results.
src/diffusers/pipelines/ideogram4 · high confidence
Introduce InstructPix2Pix training examples for Stable Diffusion and SDXL
Adds new training scripts (\train\_instruct\_pix2pix.py\ and \train\_instruct\_pix2pix\_sdxl.py\) along with comprehensive documentation and tests, enabling users to fine-tune Stable Diffusion and Stable Diffusion XL models to follow image edit instructions. The SDXL variant includes support for VAE precision options to stabilize training, while both scripts provide multi-GPU support, validation logging via Weights & Biases, and checkpoint management.
_examples/instruct\pix2pix · high confidence
Introduce JoyImage Edit and Edit Plus pipelines for image editing
Added the JoyImage Edit and JoyImage Edit Plus pipelines to the diffusers library, enabling users to perform instruction-guided image editing. The JoyImage Edit pipeline supports single-image editing using a Qwen3-VL text encoder and a 3-D transformer, while the JoyImage Edit Plus pipeline extends this to multi-image instruction-guided editing, allowing multiple reference images to be processed independently before joint denoising. Both pipelines include a custom image processor that handles bucket-based resolution selection and resize-center-crop preprocessing, and are exported via the \src/diffusers/pipelines/joyimage\ module.
src/diffusers/pipelines/joyimage · high confidence
Introduce Kandinsky 3.0 text-to-image and image-to-image pipelines
Adds the Kandinsky3Pipeline for text-to-image generation and Kandinsky3Img2ImgPipeline for image-to-image generation, both leveraging the Kandinsky3UNet, T5 text encoder, and VQModel. The text-to-image pipeline supports classifier-free guidance, negative prompts, and LoRA loading via StableDiffusionLoraLoaderMixin, while the image-to-image pipeline adds image preprocessing and strength-based timestep scheduling. A conversion script is also provided to migrate existing U-Net state dictionaries to the Kandinsky3UNet format, and the module exposes lazy imports for optional dependency handling.
src/diffusers/pipelines/kandinsky3 · high confidence
Introduce Kandinsky 5.0 pipelines for text-to-image, image-to-image, text-to-video, and image-to-video generation
The Kandinsky 5.0 model family is now available in Diffusers, providing four new pipelines: \Kandinsky5T2IPipeline\ for text-to-image, \Kandinsky5I2IPipeline\ for image-to-image, \Kandinsky5T2VPipeline\ for text-to-video, and \Kandinsky5I2VPipeline\ for image-to-video. These pipelines utilize the \Kandinsky5Transformer3DModel\ and support both the \AutoencoderKLHunyuanVideo\ (for video tasks) and \AutoencoderKL\ (for image tasks) VAEs. They are built on the \Qwen2.5-VL\ text encoder and \CLIP\ text encoder, and include support for LoRA loading via \KandinskyLoraLoaderMixin\.
src/diffusers/pipelines/kandinsky5 · high confidence
Introduce Kandinsky v2.2 pipeline suite
Adds the Kandinsky v2.2 pipeline components to the library, including the core \KandinskyV22Pipeline\ for text-to-image generation, \KandinskyV22PriorPipeline\ for creating image embeddings, and specialized variants for image-to-image (\KandinskyV22Img2ImgPipeline\), inpainting (\KandinskyV22InpaintPipeline\), and ControlNet-guided generation (\KandinskyV22ControlnetPipeline\). The update also introduces combined pipelines (\KandinskyV22CombinedPipeline\, etc.) that wrap these individual steps for easier end-to-end usage, along with the necessary lazy-import structure in \\_\init\\_.py\ to manage dependencies.
_src/diffusers/pipelines/kandinsky2\2 · high confidence
Introduce LTX Video pipeline suite and latent upsampler
Adds a new LTX Video pipeline module under \src/diffusers/pipelines/ltx\ that provides a complete set of generation tools for the LTX-Video model. This includes the base \LTXPipeline\ for text-to-video, \LTXConditionPipeline\ for flexible frame conditioning (images or video segments), \LTXImageToVideoPipeline\ for image-to-video, and \LTXI2VLongMultiPromptPipeline\ for long-form multi-prompt generation with temporal tiling. The module also introduces \LTXLatentUpsamplePipeline\ and the \LTXLatentUpsamplerModel\ to handle spatial and temporal upscaling of VAE latents, along with the necessary \LTXPipelineOutput\ data class and lazy-loading infrastructure.
src/diffusers/pipelines/ltx · high confidence
Introduce LTX-2 video generation pipelines and supporting components
Adds the LTX-2 pipeline module (\src/diffusers/pipelines/ltx2\) with a full suite of video generation capabilities, including a base text-to-video pipeline (\LTX2Pipeline\), image-to-video (\LTX2ImageToVideoPipeline\), condition-based generation (\LTX2ConditionPipeline\), and advanced workflows like in-context LoRA (\LTX2InContextPipeline\), HDR video (\LTX2HDRPipeline\), and Distilled Frame Refinement (\LTX2DFRPipeline\). The module also introduces the underlying model components required for these pipelines, such as text connectors (\LTX2TextConnectors\), a duration prediction head (\LTX2DurationHead\), a latent upsampler (\LTX2LatentUpsamplerModel\), and HDR-specific image processing (\LTX2VideoHDRProcessor\), along with utilities for video export and latent layout planning.
src/diffusers/pipelines/ltx2 · high confidence
Introduce Latent Diffusion pipelines for text-to-image and super-resolution
Users can now generate images from text prompts or upscale low-resolution images using the new \LDMTextToImagePipeline\ and \LDMSuperResolutionPipeline\. The text-to-image pipeline supports configurable image dimensions, guidance scale, and optional pre-generated latents, while the super-resolution pipeline accepts PIL images or tensors and supports multiple schedulers including DDIM, PNDM, and LMS. Both pipelines are exposed via the \diffusers.pipelines.latent\_diffusion\ module and support lazy imports for faster startup times.
_src/diffusers/pipelines/latent\diffusion · high confidence
Introduce LattePipeline for text-to-video generation
Added a new \LattePipeline\ in the \diffusers\ library to support text-to-video generation using the Latte (Latent Diffusion Transformer) architecture. This new pipeline integrates a T5 text encoder, an AutoencoderKL for video encoding/decoding, and the \LatteTransformer3DModel\ to denoise latents, enabling users to generate video content from text prompts.
src/diffusers/pipelines/latte · high confidence
Introduce LongCat-Image and LongCat-Image Edit pipelines
Added the LongCat-Image pipeline for text-to-image generation and the LongCat-Image Edit pipeline for image editing tasks. These new pipelines integrate the LongCatImageTransformer2DModel and leverage Qwen2.5-VL and Qwen2 models for prompt understanding and processing, enabling users to generate and edit images using the LongCat-Image architecture.
_src/diffusers/pipelines/longcat\image · high confidence
Introduce LongCatAudioDiT pipeline for text-to-audio generation
Users can now generate audio from text prompts using the new LongCatAudioDiTPipeline. This pipeline integrates a UMT5 text encoder, a LongCatAudioDiT transformer, and a VAE to synthesize audio files, supporting features like duration estimation from text and configurable inference steps.
_src/diffusers/pipelines/longcat\_audio\dit · high confidence
Introduce Mochi text-to-video pipeline
Adds the MochiPipeline for text-to-video generation, including the main pipeline implementation, a dedicated MochiPipelineOutput class, and the module initialization. The pipeline integrates the MochiTransformer3DModel, AutoencoderKLMochi, T5 text encoder, and FlowMatchEulerDiscreteScheduler, and supports LoRA loading via Mochi1LoraLoaderMixin.
src/diffusers/pipelines/allegro, src/diffusers/pipelines/mochi · high confidence
Introduce Modular Diffusers pipelines for composable model architectures
This release introduces the Modular Diffusers framework, allowing users to build and run pipelines by composing individual model blocks (such as UNets, VAEs, and text encoders) rather than loading monolithic pipeline classes. The \src/diffusers/modular\_pipelines\ module provides core infrastructure including \ModularPipeline\, \AutoPipelineBlocks\, and a \ComponentsManager\ that handles component loading, configuration, and custom offloading strategies. It ships with pre-configured modular pipelines for a wide range of models, including Stable Diffusion XL, Stable Diffusion 3, Flux, Flux 2 (with Klein support), Wan (image-to-video and animate), Helios, Ideogram 4, Krea 2, Qwen Image, LTX, LTX 2.5, Cosmos 3, Ernie Image, Hunyuan Video 1.5, MiniMax H3, MiniMax Music 3, Z-Image, and Anima. The system also supports integration with the Mellon node-based workflow editor and allows saving modular pipelines to the Hugging Face Hub.
_src/diffusers/modular\pipelines · high confidence
Introduce Motif-Video pipelines for text-to-video and image-to-video generation
Added the Motif-Video pipeline components, including \MotifVideoPipeline\ for text-to-video generation and \MotifVideoImage2VideoPipeline\ for image-to-video generation. These pipelines leverage the \MotifVideoTransformer3DModel\, \AutoencoderKLWan\ VAE, and \T5Gemma2Encoder\ text encoder (requiring transformers \>= 5.1.0) to generate video content from text prompts or initial images, with support for XLA and standard guidance methods.
_src/diffusers/pipelines/motif\video · high confidence
Introduce ONNX Stable Diffusion pipelines with deprecation warnings
The \src/diffusers/pipelines/stable\_diffusion\ module now includes ONNX-based pipelines (\OnnxStableDiffusionPipeline\, \OnnxStableDiffusionImg2ImgPipeline\, \OnnxStableDiffusionInpaintPipeline\, and \OnnxStableDiffusionUpscalePipeline\) alongside the standard PyTorch implementations. These ONNX pipelines are marked as deprecated (with \\_last\_supported\_version\ set to \0.43.0\) and will be removed in diffusers 1.0.0. The module also exports the \CLIPImageProjection\ model and the \StableDiffusionPipelineOutput\ class, and provides a conversion script (\convert\_from\_ckpt.py\) to migrate original Stable Diffusion checkpoints to the diffusers format.
_src/diffusers/pipelines/stable\diffusion · high confidence
Introduce OmniGen multimodal-to-image generation pipeline
Adds the OmniGen pipeline, enabling users to generate images from multimodal inputs (text instructions and reference images). This new capability includes the \OmniGenPipeline\ class for orchestration, an \OmniGenMultiModalProcessor\ for handling image preprocessing and tokenization, and the necessary module exports in \\_\init\\_.py\.
src/diffusers/pipelines/omnigen · high confidence
Introduce PRX and PRXPixel text-to-image pipelines
Adds the \PRXPipeline\ and \PRXPixelPipeline\ classes to the \diffusers\ library, enabling text-to-image generation using Photoroom's PRX models. The standard \PRXPipeline\ supports resolutions up to 1024px with predefined aspect-ratio bins and uses a T5-Gemma text encoder, while the new \PRXPixelPipeline\ operates directly in pixel space (bypassing the VAE) with a default 1024px resolution and a Qwen3-VL text encoder. The module also includes a custom \T5GemmaEncoder\ wrapper to ensure compatibility with transformers 5.x composite configs and defines a \PRXPipelineOutput\ dataclass for standardized results.
src/diffusers/pipelines/prx · high confidence
Introduce PixArt-Alpha and PixArt-Sigma pipelines
Adds two new text-to-image generation pipelines, \PixArtAlphaPipeline\ and \PixArtSigmaPipeline\, to the \diffusers\ library. These pipelines support the PixArt-Alpha and PixArt-Sigma models, featuring resolution binning for various aspect ratios (256px, 512px, 1024px, and 2048px) and integration with T5 text encoders. The implementation includes support for custom timesteps and sigmas, as well as compatibility with PyTorch/XLA.
_src/diffusers/pipelines/pixart\alpha · high confidence
Introduce Qwen-Image pipeline family
Adds a new \qwenimage\ module exposing a suite of pipelines for the Qwen-Image model, including \QwenImagePipeline\ for text-to-image, \QwenImageEditPipeline\ and \QwenImageEditPlusPipeline\ for image editing, \QwenImageInpaintPipeline\ and \QwenImageEditInpaintPipeline\ for inpainting, \QwenImageImg2ImgPipeline\ for image-to-image, and \QwenImageControlNetPipeline\ and \QwenImageControlNetInpaintPipeline\ for ControlNet-guided generation. The module also provides the \QwenImagePipelineOutput\ dataclass for standardized results and registers all pipelines in the package's lazy import structure.
src/diffusers/pipelines/qwenimage · high confidence
Introduce Stable Audio 3 pipelines and duration embedder
Adds the Stable Audio 3 pipeline suite (\StableAudio3Pipeline\, \StableAudio3AudioToAudioPipeline\, \StableAudio3InpaintPipeline\) and the \StableAudio3DurationEmbedder\ module. The pipelines implement text-to-audio, audio-to-audio variation, and inpainting capabilities using a distilled rectified-flow DiT, featuring a single \duration\ parameter (in seconds) instead of start/end times, and defaulting \guidance\_scale\ to 1.0 for the distilled checkpoint. The duration embedder encodes audio length into a global conditioning vector for the model.
_src/diffusers/pipelines/stable\_audio\3 · high confidence
Introduce Stable Audio text-to-audio generation pipeline
Adds a new \StableAudioPipeline\ for generating audio from text prompts, including the \StableAudioProjectionModel\ for conditioning and the necessary module exports. The pipeline leverages a T5 text encoder, an AutoencoderOobleck VAE, and a StableAudioDiT transformer, supporting features like negative prompts, configurable audio duration, and Torch XLA acceleration.
_src/diffusers/pipelines/stable\audio · high confidence
Introduce Stable Diffusion 3 ControlNet pipelines
Added \StableDiffusion3ControlNetPipeline\ and \StableDiffusion3ControlNetInpaintingPipeline\ to the \diffusers\ library, enabling users to apply ControlNet conditioning to Stable Diffusion 3 models. These new pipelines support single and multiple ControlNets, IP-Adapter integration, and LoRA loading, allowing for precise image generation and inpainting guided by external control signals.
_src/diffusers/pipelines/controlnet\sd3 · high confidence
Introduce Stable Diffusion XL pipeline suite
This change adds the core Stable Diffusion XL pipelines to the library, providing \StableDiffusionXLPipeline\ for text-to-image generation, \StableDiffusionXLImg2ImgPipeline\ for image-to-image editing, \StableDiffusionXLInpaintPipeline\ for inpainting, and \StableDiffusionXLInstructPix2PixPipeline\ for instruction-based image editing. The implementation includes a dedicated \StableDiffusionXLPipelineOutput\ dataclass, optional invisible watermarking support via the \StableDiffusionXLWatermarker\, and integration with LoRA, IP-Adapter, and Textual Inversion loading mixins. These pipelines are exposed through the \src/diffusers/pipelines/stable\_diffusion\_xl\ module with lazy imports to optimize startup performance.
_src/diffusers/pipelines/stable\_diffusion\_3, src/diffusers/pipelines/stable\_diffusion\xl · high confidence
Introduce Stable Video Diffusion pipeline
Adds the \StableVideoDiffusionPipeline\ and its output class, enabling users to generate video from a single input image. The implementation integrates a temporal VAE, a CLIP image encoder, and a spatio-temporal UNet, while utilizing a new \VideoProcessor\ for resizing and frame handling.
_src/diffusers/pipelines/stable\_video\diffusion · high confidence
Introduce T2I-Adapter pipelines for Stable Diffusion and SDXL
Adds \StableDiffusionAdapterPipeline\ and \StableDiffusionXLAdapterPipeline\ to the \diffusers\ library, enabling text-to-image generation conditioned on structural inputs (such as sketches, depth maps, or color palettes) via the T2I-Adapter architecture. These new pipelines integrate with existing features like LoRA loading, single-file model loading, and PyTorch/XLA support, allowing users to apply adapter models to guide the diffusion process while maintaining compatibility with standard Stable Diffusion workflows.
_src/diffusers/pipelines/t2i\adapter · high confidence
Introduce Textual Inversion training examples for Stable Diffusion and SDXL
The \examples/textual\_inversion\ directory now provides complete, runnable training scripts (\textual\_inversion.py\ for Stable Diffusion v1/v2 and \textual\_inversion\_sdxl.py\ for SDXL) along with PyTorch and Flax/JAX variants. These scripts enable users to fine-tune text-to-image models on custom datasets using the textual inversion method, supporting features like multi-vector embeddings, safetensors serialization, checkpointing, and validation image generation. Documentation (\README.md\, \README\_sdxl.md\) and automated tests are included to guide setup and verify training and checkpointing behavior.
_examples/textual\inversion · high confidence
Introduce TorchAO quantizer with safetensors support
Adds a new TorchAO quantizer implementation (\TorchAoHfQuantizer\) to the diffusers library, enabling users to load and run models quantized with the TorchAO library. This change introduces support for saving and loading TorchAO-serialized tensors via safetensors (requiring torchao \>= 0.16.0 for full serialization support) and enforces a minimum torchao version of 0.15.0. The implementation includes logic to handle safe globals for torch serialization and resolves quantization configurations for modules.
src/diffusers/quantizers/torchao · high confidence
Introduce Wan video generation pipelines
Adds a new \src/diffusers/pipelines/wan\ module providing five pipelines for the Wan model family: \WanPipeline\ (text-to-video), \WanImageToVideoPipeline\ (image-to-video), \WanVideoToVideoPipeline\ (video-to-video), \WanAnimatePipeline\ (character animation/replacement), and \WanVACEPipeline\ (controllable generation with first/last frames and masks). The module includes a shared \WanPipelineOutput\ dataclass, a \WanAnimateImageProcessor\ for reference image preprocessing, and lazy-loading wiring in \\_\init\\_.py\ so these pipelines are available via \from diffusers.pipelines.wan import ...\ when PyTorch and Transformers are installed.
src/diffusers/pipelines/animatediff, src/diffusers/pipelines/helios, src/diffusers/pipelines/wan · high confidence
Introduce Z-Image modular pipeline support
Adds a new modular pipeline implementation for the Z-Image model, enabling users to compose and execute generation workflows using modular blocks. This includes the \ZImageModularPipeline\ class, which integrates with \ZImageLoraLoaderMixin\, and a set of pipeline blocks for text encoding (\ZImageTextEncoderStep\), denoising (\ZImageDenoiseStep\, \ZImageLoopDenoiser\), and decoding (\ZImageVaeDecoderStep\). The implementation supports both text-to-image and image-to-image workflows via \ZImageCoreDenoiseStep\ and \ZImageImage2ImageCoreDenoiseStep\, handling latent preparation, timestep setting, and guidance logic through the modular architecture.
_src/diffusers/modular\_pipelines/z\image · high confidence
Introduce diffusers-cli with environment, run, and modular pipeline commands
The \diffusers-cli\ tool is now available with a structured command set. The \env\ command provides a comprehensive diagnostic report, including versions for diffusers, PyTorch, transformers, accelerate, PEFT, xformers, and quantization backends (bitsandbytes, gguf, optimum-quanto, torchao, nvidia-modelopt), along with platform and accelerator details. The \run\ command serves as a single agentic entry point to execute standard and modular pipelines, supporting features like LoRA adapters, CPU offloading, and remote execution via Hugging Face Sandboxes. A new \custom\_blocks\ command allows users to package local \ModularPipelineBlocks\ subclasses for the Hub. Additionally, the \schema\ command prints input schemas for pipelines without downloading weights, and the \skills\ command installs AI agent skill bundles. The previously available \fp16\_safetensors\ command is now deprecated.
src/diffusers/commands · high confidence
Introduce experimental parallelism and modular model components
This change introduces experimental support for distributed inference via Context Parallelism (Ring and Ulysses attention) and Tensor Parallelism, accessible through new configuration classes in \src/diffusers/models/\_modeling\_parallel.py\. It also restructures the codebase by extracting activation functions into a dedicated \src/diffusers/models/activations.py\ module and consolidating attention logic into \src/diffusers/models/attention.py\ and \src/diffusers/models/attention\_dispatch.py\. Additionally, a new \src/diffusers/experimental\ package is added to host experimental features, currently including a Reinforcement Learning pipeline.
src/diffusers/models · high confidence
Introduce modular Stable Diffusion XL pipeline components
Adds a new modular pipeline implementation for Stable Diffusion XL, exposing \StableDiffusionXLModularPipeline\ and \StableDiffusionXLAutoBlocks\ to allow users to compose generation workflows from granular steps (such as text encoding, latent preparation, denoising, and decoding) rather than using a single monolithic call. This change also deprecates the \upcast\_vae\ helper in favor of directly casting the VAE to float32, aligning with the new modular architecture's explicit component management.
_src/diffusers/modular\_pipelines/flux2, src/diffusers/modular\_pipelines/stable\_diffusion\xl · high confidence
Introduce modular hook system for advanced inference optimizations
A new \src/diffusers/hooks\ module has been added, providing a unified registry and application layer for advanced inference optimizations. This system introduces dedicated hooks for Context Parallelism, FasterCache, First Block Cache, Group Offloading (including disk/NVMe support and TorchAO compatibility), Layer Skip, Layerwise Casting, MagCache, Pyramid Attention Broadcast, SeaCache, TaylorSeer Cache, and Tensor Parallelism. Users can now apply these performance-enhancing techniques to their models through a consistent, modular interface.
src/diffusers/hooks · high confidence
Introduce modular pipeline architecture for Cosmos3
Adds a new modular pipeline implementation for the Cosmos3 model, breaking the generation process into discrete, composable blocks (e.g., text encoding, vision preparation, denoising, decoding) rather than a single monolithic script. This change introduces \Cosmos3OmniModularPipeline\ and \Cosmos3DistilledModularPipeline\ classes, along with supporting modules for handling video/audio decoding, control signal transfer, and action generation, enabling more flexible and reusable pipeline construction for Cosmos3 workflows.
_src/diffusers/modular\pipelines/cosmos · high confidence
Introduce modular pipeline blocks for Wan 2.1 and Wan 2.2 video models
This change adds a new modular pipeline implementation for the Wan family of video generation models, located in \src/diffusers/modular\pipelines/wan\. It introduces reusable pipeline blocks for text encoding, denoising, and decoding, supporting both Wan 2.1 and Wan 2.2 architectures. The module provides specific block sets for text-to-video (\WanBlocks\, \Wan22Blocks\), image-to-video (\Wan22Image2VideoBlocks\, \WanImage2VideoAutoBlocks\), and video-to-video/animation tasks (\Wan22VaceBlocks\). These blocks are exposed via the \\\init\\_.py\ module and can be used to construct custom workflows for these models.
_src/diffusers/modular\pipelines/wan · high confidence
Introduce modular pipeline support for Flux and Flux Kontext
Users can now compose Flux and Flux Kontext image generation workflows using the modular pipeline framework. This change adds the \FluxModularPipeline\ and \FluxKontextModularPipeline\ classes, along with a suite of reusable blocks for text encoding, image preprocessing, latent preparation, denoising, and decoding. This enables advanced use cases such as custom workflow construction, intermediate state inspection, and fine-grained control over the generation process for both text-to-image and image-to-image tasks.
_src/diffusers/modular\pipelines/flux · high confidence
Introduce modular pipeline support for LTX-2 and LTX-2.5
Adds a new modular pipeline implementation for the LTX-2 and LTX-2.5 video generation models. This change introduces \LTX2ModularPipeline\ and \LTX25ModularPipeline\ classes, along with a comprehensive set of modular blocks for encoding, denoising, and decoding. Key features include support for joint video and audio generation, a new LTX-2.5-specific diffusion-based video decoder, and an optional prompt enhancement step powered by a Gemma-4 model. The implementation also handles various conditioning workflows, including text-to-video, image-to-video, and in-context conditioning. Also in this area: 4 commits (0 fixes) modifying src/diffusers.
_src/diffusers/modular\pipelines/ltx2 · medium confidence · unverified
Introduce unified quantization framework with pipeline-level configuration
Diffusers now includes a dedicated \quantizers\ module that provides a unified, auto-dispatching system for loading and applying model quantization. This change introduces the \DiffusersAutoQuantizer\ to automatically select the correct backend (such as BitsAndBytes, TorchAO, GGUF, Quanto, or AutoRound) based on the model's configuration, and adds \PipelineQuantizationConfig\ to allow users to apply quantization on-the-fly to specific pipeline components during \from\_pretrained\. The existing \Quanto\ backend is marked as deprecated in favor of these newer, more flexible backends.
src/diffusers/quantizers · high confidence
Introduction of FreeInit and FreeNoise optimization utilities for pipelines
This change adds new utility modules, \free\_init\_utils.py\ and \free\_noise\_utils.py\, to the \src/diffusers/pipelines\ directory. These modules introduce the \FreeInitMixin\ and \AnimateDiffFreeNoiseMixin\ classes, which provide mechanisms to optimize inference speed and quality. \FreeInit\ allows users to enable noise re-initialization with configurable filtering methods (Butterworth, ideal, Gaussian) and sampling strategies to accelerate convergence. \FreeNoise\ provides logic to split inference for large tensors and replace standard transformer blocks with \FreeNoiseTransformerBlock\s in AnimateDiff pipelines to reduce memory usage and improve performance.
src/diffusers/pipelines · high confidence
Introduction of new schedulers and lazy import infrastructure
The schedulers module now includes several new scheduler implementations: AmusedScheduler for masked token generation, BlockRefinementScheduler for iterative block-wise refinement, ConsistencyDecoderScheduler for consistency model decoding, CMStochasticIterativeScheduler for consistency models, and CosineDPMSolverMultistepScheduler for cosine-based noise schedules. To support this expanded API, the module's \_\init\\_.py has been refactored to use lazy imports, ensuring optional dependencies like scipy and torchsde are only loaded when their specific schedulers are accessed.
src/diffusers/schedulers · high confidence
Introduction of the ControlNet module with support for multiple model architectures
A new \controlnets\ module has been added to the library, centralizing the implementation of ControlNet models. This change introduces dedicated model classes for several architectures, including the base \ControlNetModel\, as well as specialized variants for Flux (\FluxControlNetModel\), SD3 (\SD3ControlNetModel\), Hunyuan (\HunyuanDiT2DControlNetModel\), Qwen-Image (\QwenImageControlNetModel\), Sana (\SanaControlNetModel\), and Cosmos (\CosmosControlNetModel\). The module also includes adapters for ControlNet Union (\ControlNetUnionModel\), ControlNet XS (\ControlNetXSAdapter\), and Sparse ControlNet (\SparseControlNetModel\), along with multi-controlnet support (\MultiControlNetModel\, \MultiControlNetUnionModel\). Users can now import and utilize these specific ControlNet implementations directly from \diffusers.models.controlnets\.
src/diffusers/models/controlnets · high confidence
Marigold pipeline expanded to support intrinsic image decomposition alongside depth and normals
The Marigold pipeline module now includes a new \MarigoldIntrinsicsPipeline\ and \MarigoldIntrinsicsOutput\ class, enabling intrinsic image decomposition (predicting albedo, roughness, metallicity, shading, and residual maps) in addition to the existing monocular depth and normals estimation capabilities. This update also introduces the \MarigoldImageProcessor\ for unified image handling and adds support for the v1-1 model variants, while maintaining compatibility with PyTorch/XLA and allowing the pipeline to operate without a VAE.
src/diffusers/pipelines/marigold · high confidence
New ACE-Step pipeline for text-to-music generation
Added the AceStepPipeline and its supporting model components (AceStepConditionEncoder, AceStepAudioTokenizer, etc.) to enable text-to-music generation. The pipeline supports multiple task types including text2music, cover, repaint, extract, lego, and complete, and integrates with the existing LoRA loading infrastructure via AceStepLoraLoaderMixin.
_src/diffusers/pipelines/ace\step · high confidence
New Amused training example with LoRA and 8-bit optimizer support
Added a new training example for the Amused model in the \examples/amused\ directory, including \train\_amused.py\ and \README.md\. This example provides recipes for fine-tuning Amused on datasets like nouns and Minecraft, supporting full fine-tuning, LoRA, and 8-bit Adam optimizers to enable training on low-memory configurations (as low as 5.5 GB).
examples/amused · high confidence
New AutoencoderKL training example for CIFAR10 and ImageNet
Users can now train a custom AutoencoderKL model using the new \train\_autoencoderkl.py\ script located in the \examples/research\_projects/autoencoderkl\ directory. The example supports training on CIFAR10 and ImageNet datasets, allowing users to fine-tune the VAE from a pretrained base model (such as \stabilityai/sd-vae-ft-mse\) with configurable parameters for resolution, learning rate, and validation images.
_examples/research\projects/autoencoderkl · high confidence
New CogVideoX LoRA training examples for text-to-video and image-to-video
Added new example scripts (\train\_cogvideox\_lora.py\ and \train\_cogvideox\_image\_to\_video\_lora.py\) and documentation for performing LoRA fine-tuning on CogVideoX models. The text-to-video script supports distributed VAE video encoding across processes to optimize memory usage during training, while the image-to-video variant includes support for validation images. Both scripts provide comprehensive guidance on data preparation, including support for line-separated text/video files and CSV formats, and detail the required environment setup with specific dependencies like \peft\>=0.6.0\ and \diffusers\.
examples/cogvideo · high confidence
New ControlNet training examples for FLUX, SD3/3.5, and SDXL
The \examples/controlnet\ directory now includes dedicated training scripts and documentation for Stable Diffusion XL (SDXL), Stable Diffusion 3/3.5 (SD3/3.5), and FLUX models, in addition to the existing Stable Diffusion 1.5/2.1 support. Users can now train ControlNet adapters for these newer architectures using the provided \train\_controlnet\_sdxl.py\, \train\_controlnet\_sd3.py\, and \train\_controlnet\_flux.py\ scripts, each accompanied by specific \README\ files detailing installation, dataset preparation, and inference usage.
examples/controlnet · high confidence
New DreamBooth training documentation for FLUX.2, Ideogram 4, Krea 2, HiDream, Lumina 2, and Qwen Image
The examples/dreambooth directory now includes dedicated README guides for training LoRA adapters on several new model families: FLUX.2 \[dev\] and FLUX 2 \[klein\] (with memory optimization strategies like remote text encoders, FSDP, and QLoRA), Ideogram 4 (supporting JSON captions and FP8/QLoRA bases), Krea 2 (with RAW-to-Turbo validation workflows), HiDream Image, Lumina 2, and Qwen Image. These guides provide specific command-line examples, dependency requirements, and configuration tips for each model.
examples/dreambooth · high confidence
New Dreambooth inpainting training examples
Added \train\_dreambooth\_inpaint.py\ and \train\_dreambooth\_inpaint\_lora.py\ scripts to the \examples/research\_projects/dreambooth\_inpaint\ directory, along with a \README.md\ documenting their usage. These scripts enable fine-tuning Stable Diffusion inpainting models using Dreambooth, supporting features such as prior-preservation loss, gradient checkpointing, 8-bit optimizers, and optional text encoder training. The LoRA variant includes support for Low-Rank Adaptation to reduce memory usage during training.
_examples/research\_projects/dreambooth\inpaint · high confidence
New Flux Control LoRA and full fine-tuning training examples
Added \train\_control\_lora\_flux.py\ and \train\_control\_flux.py\ scripts to the \examples/flux-control\ directory, enabling users to train Control LoRAs or perform full fine-tuning on the Flux.1-dev model using structural conditions like depth maps or poses. The examples demonstrate expanding the model's input channels to incorporate control latents, with support for features such as gradient checkpointing, mixed precision (bf16), DeepSpeed Zero-2 optimization, and validation logging to Weights & Biases or TensorBoard.
examples/flux-control · high confidence
New Flux.1 Dev LoRA fine-tuning example with NF4 quantization
Added a new research example in \examples/research\_projects/flux\_lora\_quantization\ that demonstrates how to fine-tune the Flux.1 Dev model using LoRA and NF4 quantization via bitsandbytes. The directory includes a training script (\train\_dreambooth\_lora\_flux\_miniature.py\) that supports memory-optimized workflows such as precomputed text embeddings, 8-bit Adam, gradient checkpointing, and DeepSpeed Zero2. It also provides a helper script (\compute\_embeddings.py\) to serialize text embeddings into a parquet file and configuration files for Accelerate and DeepSpeed, along with documentation on merging and dequantizing LoRA weights for inference.
_examples/research\_projects/flux\_lora\quantization · high confidence
New InstructPix2Pix LoRA fine-tuning example
Added a new training script and documentation for fine-tuning Stable Diffusion InstructPix2Pix using LoRA adapters. This example allows users to customize image editing models by training on custom datasets (such as cartoonization) with support for PEFT, gradient checkpointing, and validation logging via Weights & Biases.
_examples/research\_projects/instructpix2pix\lora · high confidence
New NVIDIA Cosmos video generation pipelines and FP8 mixed-precision support
This update introduces a new \cosmos\ pipeline package providing inference support for NVIDIA Cosmos models. It includes \Cosmos2TextToImagePipeline\ and \Cosmos2VideoToWorldPipeline\ for the Cosmos Predict2 series, \Cosmos2\_5\_PredictBasePipeline\ and \Cosmos2\_5\_TransferPipeline\ for the Predict2.5 and Transfer2.5 series (supporting control inputs like edge, depth, and segmentation maps), and \Cosmos3OmniPipeline\ for the Cosmos3 series. The Cosmos3 pipeline adds support for mixed W8A8/W8A16 denoising to optimize performance with ModelOpt FP8 checkpoints, and the Transfer2.5 pipeline enables auto-regressive inference for video generation.
src/diffusers/pipelines/cosmos · high confidence
New ONNX Runtime Textual Inversion example
Added a new training script and documentation for performing Textual Inversion fine-tuning using ONNX Runtime to accelerate the process. The \textual\_inversion.py\ script demonstrates how to adapt the standard Stable Diffusion training procedure for ONNX, including specific handling for FP16 optimization and validation loops, while the accompanying README provides setup instructions and usage examples for both local and Colab environments.
_examples/research\_projects/onnxruntime/textual\inversion · high confidence
New PAG pipelines for Stable Diffusion, SDXL, HunyuanDiT, Kolors, and more
Users can now enable Perturbed Attention Guidance (PAG) across a wide range of diffusion pipelines by passing \enable\_pag=True\ during loading. This location introduces the \pag\ subpackage, which provides the \PAGMixin\ utility and specific pipeline classes—including \StableDiffusionPAGPipeline\, \StableDiffusionXLControlNetPAGPipeline\, \HunyuanDiTPAGPipeline\, and \KolorsPAGPipeline\—that apply attention guidance to improve image quality. Users can configure which layers receive guidance via the \pag\_applied\_layers\ argument and adjust the guidance strength with \pag\_scale\.
src/diffusers/pipelines/pag · high confidence
New SANA-Video pipelines for text-to-video and image-to-video generation
Added the \SanaVideoPipeline\ for text-to-video generation and the \SanaImageToVideoPipeline\ for image-to-video generation within the \diffusers\ library. These pipelines support the SANA model architecture, allowing users to generate video content from text prompts or animate static images, with support for aspect ratio binning and LoRA loading.
_src/diffusers/pipelines/sana\video · high confidence
New SD3 DreamBooth LoRA example for low-memory Colab environments
Added a new educational example in \examples/research\_projects/sd3\_lora\_colab\ that enables Stable Diffusion 3 DreamBooth LoRA fine-tuning on free-tier Google Colab instances with under 16GB of GPU VRAM. The entry includes a Colab notebook (\sd3\_dreambooth\_lora\_16gb.ipynb\), a Python script for pre-computing and serializing text embeddings (\compute\_embeddings.py\) using 8-bit T5 encoding to reduce memory usage, and a miniature training script (\train\_dreambooth\_lora\_sd3\_miniature.py\) that utilizes 8-bit Adam optimization, gradient checkpointing, and FP16 precision to fit within memory constraints.
_examples/research\_projects/sd3\_lora\colab · high confidence
New SDXL ControlNet training script for WebDataset
Added a new training script (\train\_controlnet\_webdataset.py\) for Stable Diffusion XL ControlNet models, enabling training with WebDataset formats. The script supports Canny and depth conditioning (using DPTImageProcessor), integrates with the Hugging Face Hub for model uploading, and includes data filtering capabilities.
_examples/research\projects/controlnet · high confidence
New Sana pipeline implementations
This location introduces the core Sana pipeline components, including \SanaPipeline\ for text-to-image generation, \SanaControlNetPipeline\ for controlled generation, \SanaSprintPipeline\ for the SANA-Sprint model, and \SanaSprintImg2ImgPipeline\ for image-to-image tasks. The \\_\init\\_.py\ module exposes these classes for lazy loading, while \pipeline\_output.py\ defines the \SanaPipelineOutput\ dataclass to standardize the return format of denoised images across all Sana-based pipelines.
src/diffusers/pipelines/sana · high confidence
New T2I-Adapter training example for Stable Diffusion XL
Users can now train T2I-Adapter models specifically for Stable Diffusion XL using the new \train\_t2i\_adapter\_sdxl.py\ script. This example includes comprehensive documentation in \README\_sdxl.md\ covering installation, dataset preparation (e.g., fill50k), training configuration with Accelerate, and inference setup. A corresponding test suite (\test\_t2i\_adapter.py\) ensures the training script functions correctly, while the main \README.md\ directs users to this SDXL-specific guide since SD 1.5 training is not yet supported.
_examples/t2i\adapter · high confidence
New VAE roundtrip example script
Added \vae\_roundtrip.py\ in the \examples/research\_projects/vae\ directory, which demonstrates encoding an input image into latent space and decoding it back to reconstruct the original image. The script supports both \AutoencoderKL\ and \AutoencoderTiny\ models, allowing users to visualize the original and reconstructed images side-by-side to evaluate reconstruction quality.
_examples/research\projects/vae · high confidence
New VQGAN training example with discriminator support
Adds a new VQGAN training example (\examples/vqgan\) that includes a training script (\train\_vqgan.py\), a GAN discriminator model (\discriminator.py\), and comprehensive tests (\test\_vqgan.py\). The training script supports training on datasets like CIFAR-10, allows architecture customization via config files, and includes features such as EMA, checkpointing, and validation logging to Weights & Biases or TensorBoard. The discriminator is ported from the Paella project and integrates with the VQModel for adversarial training.
examples/vqgan · high confidence
New Z-Image pipeline family for image generation, inpainting, and ControlNet conditioning
Adds a new \z\_image\ pipeline module providing \ZImagePipeline\ for text-to-image generation, \ZImageImg2ImgPipeline\ for image-to-image transformation, \ZImageInpaintPipeline\ for inpainting, \ZImageControlNetPipeline\ and \ZImageControlNetInpaintPipeline\ for ControlNet-guided generation and inpainting, and \ZImageOmniPipeline\ which integrates a SigLIP-2 vision model for enhanced conditioning. All pipelines support LoRA loading via \ZImageLoraLoaderMixin\, use the \FlowMatchEulerDiscreteScheduler\, and expose a \ZImagePipelineOutput\ dataclass for results.
_src/diffusers/pipelines/z\image · high confidence
New advanced DreamBooth LoRA training scripts for Flux, SDXL, and SD 1.5
The \examples/advanced\_diffusion\_training\ directory now includes comprehensive training scripts (\train\_dreambooth\_lora\_flux\_advanced.py\, \train\_dreambooth\_lora\_sdxl\_advanced.py\, and \train\_dreambooth\_lora\_sd15\_advanced.py\) along with their corresponding documentation and tests. These scripts provide advanced fine-tuning capabilities, including support for Pivotal Tuning (textual inversion combined with diffusion fine-tuning), DoRA (Direction-Preserving LoRA), and EDM-style training for SDXL. The Flux script specifically adds support for pivotal tuning on both CLIP and T5 text encoders, configurable LoRA target modules, and latent caching to optimize memory usage during training.
_examples/advanced\_diffusion\training · high confidence
New asynchronous server example for concurrent Stable Diffusion 3 inference
An example asynchronous server has been added to \examples/server-async\ that enables safe, concurrent inference using Stable Diffusion 3 and 3.5 models. The implementation keeps a single copy of the heavy model parameters in memory while creating per-request lightweight views to avoid race conditions and excessive memory usage. It includes a \RequestScopedPipeline\ that manages thread-safe wrappers for tokenizers, VAEs, and image processors, along with a \BaseAsyncScheduler\ and \async\_retrieve\_timesteps\ helper to isolate mutable scheduler state. The server is built with FastAPI, exposes an inference endpoint at \/api/diffusers/inference\, and provides a test client for validation.
examples/server-async · high confidence
New autoencoders module with expanded model support
The \src/diffusers/models/autoencoders\ directory has been restructured into a dedicated module, introducing a comprehensive \\_\init\\_.py\ that exposes a wide range of autoencoder implementations. This update adds support for numerous new models including Cosmos 3 (audio and video), Allegro, CogVideoX, Wan, LTX Video (0.9.1, 2.0, 2.3, 2.5), MiniMax H3, Qwen-Image (2.1), Hunyuan Video/Image, Mochi, Sana, DC-AE, RAE, VidTok, KVAE, and Stable Audio, alongside core classes like \AutoencoderKL\ and \AsymmetricAutoencoderKL\. Users can now access these specialized VAEs and tokenizers directly from the \diffusers.models.autoencoders\ namespace.
src/diffusers/models/autoencoders · high confidence
New benchmarking suite for Flux, LTX, SDXL, and Wan models
A new benchmarking infrastructure has been added to the \benchmarks\ directory, allowing users to measure latency and memory usage for popular models including Flux, LTX-Video, SDXL, and Wan. The suite provides scripts (\benchmarking\_flux.py\, \benchmarking\_ltx.py\, \benchmarking\_sdxl.py\, \benchmarking\_wan.py\) that test specific scenarios such as base bfloat16 inference, torch compilation, NF4 quantization, layerwise upcasting, and group offloading. Results are collated into CSV files and can be automatically pushed to the Hugging Face Hub via \push\_results.py\.
benchmarks · high confidence
New diffusion-based policy and locomotion examples for reinforcement learning
Added two new examples in the reinforcement learning directory: \diffusion\_policy.py\ and \run\_diffuser\_locomotion.py\, along with a \README.md\. The diffusion policy example demonstrates a robot control model that uses a diffusion model to predict smooth movement trajectories for pushing a T-shaped block, requiring specific dependencies like \free-mujoco-py\ and \gym\. The locomotion example showcases the use of \ValueGuidedRLPipeline\ from \diffusers.experimental\ to run a hopper agent in a Gym environment, allowing users to adjust guide steps for reward maximization versus faster sampling.
_examples/reinforcement\learning · high confidence
New example for distillation-based quantization of Textual Inversion models
Added a new research example in \examples/research\_projects/intel\_opts/textual\_inversion\_dfq\ that demonstrates how to apply knowledge distillation for quantization to Textual Inversion models. The entry includes \textual\_inversion.py\ for training, \text2images.py\ for inference, and a \README.md\ with step-by-step instructions. Users can now fine-tune a Stable Diffusion model using Textual Inversion and subsequently compress it to INT8 precision via quantization-aware training with distillation, resulting in a significantly smaller model (e.g., UNet size reduced from \~3.2GB to \~827MB) while maintaining generation quality.
_examples/research\_projects/intel\_opts/textual\_inversion\dfq · high confidence
New example for searching and loading models from Civitai and Hugging Face
The \examples/model\_search\ directory now includes a new example that leverages the \auto\_diffusers\ library to search for and load models directly from Civitai and the Hugging Face Hub. This addition provides \EasyPipeline\ classes (for text-to-image, image-to-image, and inpainting) that allow users to search by keyword or URL, automatically detect the appropriate pipeline type from single-file checkpoints, and load LoRA or TextualInversion weights seamlessly.
_examples/model\search · high confidence
New model conversion scripts for ACE-Step, Anima, AnyFlow, AuraFlow, Asymmetric VQGAN, BLIP Diffusion, and more
Added a suite of new conversion scripts in the \scripts/\ directory to migrate external model checkpoints into the Diffusers format. These include \convert\_ace\_step\_to\_diffusers.py\ for ACE-Step audio models, \convert\_anima\_to\_diffusers.py\ for Anima, \convert\_anyflow\_to\_diffusers.py\ for AnyFlow variants, \convert\_aura\_flow\_to\_diffusers.py\ for AuraFlow, \convert\_asymmetric\_vqgan\_to\_diffusers.py\ for Asymmetric VQGAN scales 1.5 and 2, and \convert\_blipdiffusion\_to\_diffusers.py\ for BLIP Diffusion. Additional scripts cover LDM uncond (\conversion\_ldm\_uncond.py\), MUSE (\convert\_amused.py\), AnimateDiff motion LoRA and modules, SparseCtrl, and LDM checkpoint naming changes (\change\_naming\_configs\_and\_checkpoints.py\).
scripts · high confidence
New modular guider system with multiple guidance implementations
The \src/diffusers/guiders\ package introduces a new modular architecture for diffusion model guidance, featuring a \BaseGuidance\ class and a collection of specific implementations including \ClassifierFreeGuidance\, \ClassifierFreeZeroStarGuidance\, \AdaptiveProjectedGuidance\, \AdaptiveProjectedMixGuidance\, \AutoGuidance\, \FrequencyDecoupledGuidance\, \LTX2Guidance\, \MagnitudeAwareGuidance\, \PerturbedAttentionGuidance\, \SkipLayerGuidance\, \SmoothedEnergyGuidance\, and \TangentialClassifierFreeGuidance\. This change provides users with a unified interface for applying various guidance techniques, supporting features like step-range control, momentum buffers, frequency-space decoupling, and layer-skip hooks, while maintaining backward compatibility through the \\_\init\\_.py\ exports.
src/diffusers/guiders · high confidence
New modular pipeline implementation for QwenImage models
This location introduces the modular pipeline architecture for QwenImage, QwenImage-Edit, QwenImage-Edit Plus, and QwenImage-Layered models. It provides the specific pipeline blocks (encoders, decoders, denoise loops, input processors) and the \\_\init\\_.py\ exports that allow these models to be used via the new modular pipeline system, replacing or supplementing the previous monolithic pipeline implementations.
_src/diffusers/modular\pipelines/qwenimage · high confidence
New multi-subject Dreambooth inpainting example
Added a new research example that demonstrates how to fine-tune Stable Diffusion inpainting models for multiple subjects simultaneously. The entry includes a training script (\train\_multi\_subject\_dreambooth\_inpainting.py\) that accepts multiple Hugging Face datasets as instance data directories, allowing users to personalize the model with several distinct subjects in a single run. It also provides a companion notebook (\multi\_inpaint\_dataset.ipynb\) for creating prompt-image-mask pairs and supports optional fine-tuning of the text encoder via the \--train\_text\_encoder\ flag to improve results, particularly for faces.
_examples/research\_projects/multi\_subject\_dreambooth\inpainting · high confidence
New profiling examples for DiffusionPipeline optimization
Added a new \examples/profiling\ directory containing a CLI tool (\profiling\_pipelines.py\) and helper utilities (\profiling\_utils.py\) to profile popular DiffusionPipeline implementations (Flux, Wan, LTX2, QwenImage) using \torch.profiler\. This tooling allows users to generate Chrome trace files to identify CPU-GPU synchronization bottlenecks and kernel launch delays, specifically to guide optimizations for \torch.compile\ workflows. The example includes a shell script for bulk profiling and detailed documentation on interpreting traces in Perfetto UI.
examples/profiling · high confidence
New repository quality and documentation validation utilities
Added a suite of new scripts in the \utils/\ directory to enforce code quality, documentation consistency, and repository structure. These include \check\_ai.py\ to validate agent skill links and references, \check\_config\_docstrings.py\ to ensure model configs include valid checkpoint links, \check\_copies.py\ to verify copied code remains consistent with its source, \check\_doc\_toc.py\ to manage and sort the documentation table of contents, \check\_dummies.py\ to synchronize backend-specific dummy imports, \check\_forward\_call\_docstrings.py\ to validate method signatures against docstrings, \check\_inits.py\ to analyze import structures, \check\_repo.py\ to audit model registration and testing coverage, and \check\_return\_annotations.py\ to enforce return type annotations on models and pipelines.
utils · high confidence
New transformer model implementations added to the library
The \src/diffusers/models/transformers\ module has been populated with new transformer model classes, making them available for import and use. The \\_\init\\_.py\ file now exports a wide range of models including ACE-Step, AuraFlow, CogVideoX, ConsisID, DiT, DualTransformer, and HunyuanDiT, among many others. These additions provide the underlying architecture components for various text-to-image, text-to-video, and music generation pipelines.
src/diffusers/models/transformers · high confidence
New unconditional image generation training example with precision and logging support
The \examples/unconditional\_image\_generation\ directory now includes a complete training example (\train\_unconditional.py\) for training DDPM UNet models, accompanied by documentation and tests. Users can now train on datasets like Oxford Flowers or Pokemon, or use their own data via local folders or Hugging Face Hub uploads. Key features include optional precision-preserving preprocessing for high-bit-depth images (e.g., medical TIFFs) via the \--preserve\_input\_precision\ flag, support for Weights & Biases logging, and xformers integration for memory-efficient attention. The example also supports multi-GPU training via Accelerate and includes robust checkpointing with total limit controls.
_examples/unconditional\_image\generation · high confidence
Repository initialization with standard project infrastructure
The repository is initialized with essential project files including a comprehensive .gitignore, Apache 2.0 LICENSE, CITATION.cff for academic referencing, and a SECURITY.md policy outlining vulnerability reporting procedures. The setup.py defines the package structure and runtime dependencies (such as torch, transformers, and safetensors), while the Makefile provides developer utilities for code quality checks, testing, and release management. Additionally, documentation and contribution guidelines are established through README.md, CODE\_OF\_CONDUCT.md, and symlinks to detailed AI agent and contribution guides.
(repo-wide) · high confidence
Architecture
Refactor utils package into modular submodules
The \src/diffusers/utils\ package has been reorganized from a single monolithic file into a structured set of modular submodules (e.g., \import\_utils\, \constants\, \deprecation\_utils\, \dynamic\_modules\_utils\, \export\_utils\, \hub\_utils\, \logging\, \outputs\, \peft\_utils\, \pil\_utils\, \remote\_utils\, \state\_dict\_utils\, \typing\_utils\, and \accelerate\utils\). This change improves code maintainability and separation of concerns by grouping related utilities into dedicated files, while the main \\\init\\_.py\ now acts as a centralized re-export hub for all public API components.
src/diffusers/utils · high confidence
Refactored loader module into a modular mixin architecture
The \src/diffusers/loaders\ package has been restructured from a monolithic layout into a modular system of mixins. The new \\_\init\\_.py\ exposes specialized loader classes—such as \FluxLoraLoaderMixin\, \IPAdapterMixin\, and \FromSingleFileMixin\—which are now implemented in dedicated files like \lora\_pipeline.py\, \ip\_adapter.py\, and \single\_file.py\. This change separates concerns, moving LoRA conversion utilities to \lora\_conversion\_utils.py\ and base logic to \lora\_base.py\, while introducing a unified \PeftAdapterMixin\ in \peft.py\ to standardize adapter management across different model types.
src/diffusers/loaders · high confidence
Reorganize UNet models into a dedicated \`unets\` module
The UNet model implementations (including UNet1D, UNet2D, UNet2DCondition, UNet3DCondition, and others) have been moved from the top-level \models\ directory into a new \src/diffusers/models/unets\ subpackage. This change updates the internal import paths and exposes the models via the new \src/diffusers/models/unets/\_\init\\_.py\, while maintaining the same public API for users importing from \diffusers\.
src/diffusers/models/unets · high confidence
Behavioural changes
2 commits (0 fixes) modifying src/diffusers/modular\_pipelines/p\_h3\_image
A change to existing behaviour in src/diffusers/modular\_pipelines/p\_h3\_image — 2 commits, 9 files: “\[feat\] Tensor parallelism for MiniMax-H3 (\#14609)”; “Add P-H3-Image (\#15171)” (refs \#14609, \#15171).
_src/diffusers/modular\_pipelines/p\_h3\image · medium confidence · unverified
2 commits (0 fixes) modifying src/diffusers/modular\_pipelines/qwenimage21
A change to existing behaviour in src/diffusers/modular\_pipelines/qwenimage21 — 2 commits, 9 files: “add modular pipeline for qwen-image 2.1 (\#14976)”; “when an InputParam is both required and has a default: set required = False and warn (\#14521)” (refs \#14976, \#14521).
_src/diffusers/modular\_pipelines/qwenimage21, src/diffusers/pipelines/bria\fibo2 · medium confidence · unverified
Deprecate KarrasVeScheduler and ScoreSdeVpScheduler
The \KarrasVeScheduler\ and \ScoreSdeVpScheduler\ classes have been moved to the \src/diffusers/schedulers/deprecated\ module. Users relying on these schedulers will now encounter deprecation warnings, indicating that these specific variance-expanding and variance-preserving SDE schedulers are scheduled for removal in a future release and should be migrated to supported alternatives.
src/diffusers/schedulers/deprecated · high confidence
Deprecated multi-token textual inversion research example
The multi-token textual inversion example in this directory is now deprecated and users are directed to the official textual inversion example in the main examples folder, which natively supports multi-token features. The README explicitly states this deprecation and provides migration guidance. The code files (multi\_token\_clip.py, textual\_inversion.py, textual\_inversion\_flax.py) remain present but are marked as research projects that are no longer actively maintained.
_examples/research\_projects/multi\_token\_textual\inversion · high confidence
Deprecated pipelines moved to dedicated module with lazy loading
Pipelines with low usage have been moved to the \src/diffusers/pipelines/deprecated\ directory. These pipelines are now imported via a lazy-loading mechanism that respects optional dependencies (such as \transformers\, \librosa\, or \note\_seq\), ensuring they remain available for existing users but are no longer actively tested or accepted for changes by the maintainers.
src/diffusers/pipelines/deprecated · high confidence
Deprecation of Stable Cascade pipelines
The Stable Cascade pipeline implementations (StableCascadeDecoderPipeline, StableCascadePriorPipeline, and StableCascadeCombinedPipeline) in src/diffusers/pipelines/stable\_cascade are now deprecated. The pipeline classes inherit from DeprecatedPipelineMixin and set \_last\_supported\_version to 0.35.2, indicating that these pipelines will be removed in a future release. Users should migrate to alternative image generation pipelines as Stable Cascade support is being sunset.
_src/diffusers/pipelines/stable\cascade · high confidence
Deprecation of legacy inference example scripts
The \image\_to\_image.py\ and \inpainting.py\ scripts in the \examples/inference\ directory are now deprecated and will be removed in a future version. Running these scripts now triggers a warning directing users to use the \StableDiffusionImg2ImgPipeline\ and \StableDiffusionInpaintPipeline\ classes directly from the \diffusers\ library instead. Users are advised to migrate to the official pipeline examples located in the \src/diffusers/pipelines\ folder.
examples/inference · high confidence
Diffusion ORPO example redirects to MaPO project
The training scripts for diffusion ORPO alignment in this directory are now deprecated; the README directs users to the MaPO project (mapo-t2i.github.io) for the official codebase, models, datasets, and paper.
_examples/research\_projects/diffusion\orpo · high confidence
IP-Adapter training examples moved to research projects
The IP-Adapter training scripts and documentation have been relocated to the \examples/research\_projects/ip\_adapter\ directory. This area now contains the \README.md\ and specific training tutorials for FaceID (\tutorial\_train\_faceid.py\), standard IP-Adapter (\tutorial\_train\_ip-adapter.py\), IP-Adapter Plus (\tutorial\_train\_plus.py\), and SDXL (\tutorial\_train\_sdxl.py\), providing users with the necessary code to fine-tune Stable Diffusion models using image prompts.
_examples/research\_projects/ip\adapter · high confidence
Stable Diffusion text-to-image training examples updated to v0.41.0.dev0
The training scripts and documentation in the \examples/text\_to\_image\ directory have been updated to require diffusers version 0.41.0.dev0. This update includes the \train\_text\_to\_image.py\ and \train\_text\_to\_image\_lora.py\ scripts for Stable Diffusion, as well as their SDXL counterparts (\train\_text\_to\_image\_sdxl.py\ and \train\_text\_to\_image\_lora\_sdxl.py\). The examples now support advanced training features such as Min-SNR weighting, EMA (Exponential Moving Average) tracking with CPU offloading, and DREAM training. The documentation has been expanded to cover these new options, along with improved guidance on using DeepSpeed for memory efficiency and PyTorch XLA for inference. Tests have been added to verify checkpointing behavior and LoRA weight loading for both standard and SDXL models.
_examples/text\_to\image · high confidence
Test coverage
Add pytest configuration and TaylorSeer cache tests; Introduce pytest-based test infrastructure for examples.
Dependencies
Standardized dependency requirements for all example scripts
This change introduces explicit \requirements.txt\ files for every example and research project directory (such as Dreambooth, ControlNet, Flux, and CogVideo), replacing previously implicit or missing dependency declarations. By pinning or specifying minimum versions for core libraries like \transformers\, \accelerate\, \peft\, and \torchvision\, users can now reliably install the correct environment for each specific training or inference script without encountering version conflicts or missing package errors.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 65 → 61 (-3.9)
- Rubric changed (rubric-2026.11.1 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 61 → 57 (-3.9)
- Architecture 84 → 99 (+14.2)
- Maturity 64 → 60 (-3.6)
- Readiness 70 → 69 (-1.2)
- Security 70 → 60 (-10.1)
- Performance 89 → 89 (+0.0)
Resolved (1624)
- Change coupling: pipeline_onnx_stable_diffusion.py ↔ pipeline_onnx_stable_diffusion_inpaint.py (src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py)
- Change coupling: pipeline_onnx_stable_diffusion_img2img.py ↔ pipeline_onnx_stable_diffusion_inpaint.py (src/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion_img2img.py)
- Change coupling: pipeline_onnx_stable_diffusion_inpaint_legacy.py ↔ pipeline_onnx_stable_diffusion_img2img.py (src/diffusers/pipelines/deprecated/stable_diffusion_variants/pipeline_onnx_stable_diffusion_inpaint_legacy.py)
- ConvUNetVAE.forward (cognitive 30) (scripts/convert_consistency_decoder.py)
- CoreAttention.forward (cognitive 18) (src/diffusers/pipelines/kolors/text_encoder.py)
- Duplicated block (10 lines × 11) (examples/community/fresco_v2v.py)
- Duplicated block (10 lines × 11) (examples/community/gluegen.py)
- Duplicated block (10 lines × 11) (examples/community/hd_painter.py)
- Duplicated block (10 lines × 11) (examples/community/lpw_stable_diffusion_xl.py)
- Duplicated block (10 lines × 12) (examples/community/clip_guided_stable_diffusion_img2img.py)
- Duplicated block (10 lines × 12) (examples/community/pipeline_hunyuandit_differential_img2img.py)
- Duplicated block (10 lines × 13) (examples/community/pipline_flux_fill_controlnet_Inpaint.py)
- Duplicated block (10 lines × 14) (examples/community/pipeline_flux_kontext_multiple_images.py)
- Duplicated block (10 lines × 15) (examples/community/fresco_v2v.py)
- Duplicated block (10 lines × 15) (examples/community/pipeline_flux_differential_img2img.py)
- Duplicated block (10 lines × 16) (examples/community/fresco_v2v.py)
- Duplicated block (10 lines × 17) (examples/community/ip_adapter_face_id.py)
- Duplicated block (10 lines × 2) (examples/community/lpw_stable_diffusion.py)
- Duplicated block (10 lines × 2) (examples/community/pipeline_stable_diffusion_xl_controlnet_adapter.py)
- Duplicated block (10 lines × 2) (examples/cosmos/train_cosmos_predict25_lora.py)
- …and 1604 more
New (3927)
- Critical CVE: [GHSA redacted] (examples/research_projects/realfill/requirements.txt)
- Critical CVE: [GHSA redacted] (examples/server/requirements.txt)
- Critical CVE: [GHSA redacted] (examples/server/requirements.txt)
- Critical vulnerability: [GHSA redacted] (examples/server/requirements.txt)
- Duplicated block (10 lines × 11) (examples/community/mod_controlnet_tile_sr_sdxl.py)
- Duplicated block (10 lines × 11) (src/diffusers/schedulers/scheduling_consistency_models.py)
- Duplicated block (10 lines × 13) (examples/community/llm_grounded_diffusion.py)
- Duplicated block (10 lines × 14) (examples/community/clip_guided_stable_diffusion_img2img.py)
- Duplicated block (10 lines × 14) (examples/community/lpw_stable_diffusion_xl.py)
- Duplicated block (10 lines × 14) (examples/community/lpw_stable_diffusion_xl.py)
- Duplicated block (10 lines × 15) (examples/community/fresco_v2v.py)
- Duplicated block (10 lines × 15) (examples/community/gluegen.py)
- Duplicated block (10 lines × 16) (examples/community/adaptive_mask_inpainting.py)
- Duplicated block (10 lines × 17) (examples/community/hd_painter.py)
- Duplicated block (10 lines × 19) (examples/community/fresco_v2v.py)
- Duplicated block (10 lines × 19) (examples/community/pipline_flux_fill_controlnet_Inpaint.py)
- Duplicated block (10 lines × 19) (src/diffusers/pipelines/cogvideo/pipeline_cogvideox.py)
- Duplicated block (10 lines × 2) (examples/community/pipeline_flux_rf_inversion.py)
- Duplicated block (10 lines × 2) (examples/community/pipeline_stg_mochi.py)
- Duplicated block (10 lines × 2) (src/diffusers/image_processor.py)
- …and 3907 more
Changes since last survey
- 39 commits — 31 feature/other, 8 fixes
By area
- src/diffusers — 16 commits
- docs/source — 11 commits
- .ai/references — 2 commits
- .github/ISSUE_TEMPLATE — 2 commits
- .github/workflows — 2 commits
- examples/dreambooth — 2 commits
- (root) — 1 commit
- examples/consistency_distillation — 1 commit
- scripts/smoke_test_relocated_pretrained.py — 1 commit
- tests/models — 1 commit
Notable commits
- fix: Fix replicated parameter dtypes during tensor-parallel loading (#15147)
- fix: Fix stale links to moved and renamed files (#15151)
- fix: [CI] Fix external links (#15269)
- fix: [docs] Fix 22 dead external links in docs/source/en (#15174)
- fix: [docs] Fix the PAN 2 context parallelism link after the distributed i… (#15238)
- fix: [examples] Fix gradient clipping in train_text_to_image_lora.py and train_lcm_distill_lora_sdxl.py (#15184)
- fix: fix: derive text-encoder alpha_pattern keys from the LoRA module name (#14972)
- fix: fix: prefer latest flash attn2 (#15000)
- change: Bump doc-builder to render math in docstring descriptions (#15223)
- change: Clarify LoRA adapter weight documentation and type hints (#15157)
- change: Document Cosmos3 Nano, Super, and Edge FP8 as standalone repos (#15236)
- change: MiniMax H3 Image (#15230)
- change: Pin hf-doc-builder to 0.6.0 (#15201)
- change: Replace the remaining forum links with GitHub Discussions (#15200)
- change: Reset stateful cache hooks after every ModularPipeline call (#15212)
- change: Style changes (#15198)
- change: Swap shift and scale in Flux2 LoRAs of final_layer.adaLN_modulation.1 (#15276)
- change: Update dead Kaggle Cats vs Dogs download URL in create_dataset doc (#15231)
- change: [Modular] Add PAN 2 (#15202)
- change: [agents] model integration: Hub checkpoint layout, no per-model tests… (#15268)
- …and 19 more
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
huggingface/diffusers was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 16 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 43e30028af5120d8b0868cb6975b9fad39cc4fab — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.73+preprod-52de90b74216.