mlc-ai/mlc-llm
53.9
Adequate · 4 November 2026
60.8k
lines of production code
Python
with C++, C
4
measurements over time
What this system is
Features
Add BERT model architecture and HuggingFace weight loading support
Users can now load and run BERT models within MLC LLM. This change introduces the BERT architecture implementation (including embeddings, attention layers, and encoder) and provides a parameter mapping module that converts weights from HuggingFace PyTorch and safetensors formats, including specific support for BGE models with prefixed weights.
_python/mlc\llm/model/bert · high confidence
Add CLIP and SigLIP vision encoders with GPU-accelerated image preprocessing
This change introduces the core vision model components for MLC LLM, adding \CLIPVisionModel\ and \SigLIPVisionModel\ implementations in \python/mlc\_llm/model/vision\. The CLIP encoder uses a custom \Linear\ layer that accumulates in float32 to preserve numerical accuracy, while the SigLIP encoder is configured without a CLS token and uses post-layer normalization. Additionally, a new \ImageProcessor\ module provides GPU-scheduled kernels for image resizing, center cropping, and rescaling, enabling efficient image preprocessing on non-CUDA backends as well.
_python/mlc\llm/model/vision · high confidence
Add DeepSeek model support with Multi-Lingual Expert (MoE) architecture
Users can now load and run DeepSeek models (including DeepSeek-v2) in MLC LLM. This change introduces the DeepSeek model implementation, including its configuration, attention mechanisms, and Multi-Lingual Expert (MoE) layers, along with a loader that maps HuggingFace PyTorch/Safetensor weights to MLC's internal format, handling weight concatenation for QKV and MoE gate/up projections.
_python/mlc\llm/model/deepseek · high confidence
Add GPT-2 and GPTNeoX model support
Users can now load and run GPT-2 and GPTNeoX models in MLC LLM. This change introduces the model architectures, configuration classes, and HuggingFace weight loaders for both models, enabling parameter mapping from HuggingFace PyTorch/safetensors formats and supporting tensor-parallel inference with PagedKVCache.
_python/mlc\llm/model/gpt2 · high confidence
Add GSM8K and MMLU benchmark evaluation scripts
New standalone evaluation scripts have been added to the benchmarking suite to assess model performance on the GSM8K (grade school math) and MMLU (massive multitask language understanding) datasets. The GSM8K evaluator (\gsm8k.py\) supports few-shot learning with optional chain-of-thought prompting and extracts numerical answers from model outputs. The MMLU evaluator (\mmlu.py\) assesses performance across 57 academic and professional subjects by selecting the highest-probability option (A-D) from logprobs, allowing users to run these specific benchmarks via command-line arguments.
_python/mlc\llm/bench/evaluation · high confidence
Add Gemma3 and Gemma3V model support
This change introduces the Gemma3 language model and the Gemma3V vision-language model to MLC LLM. It adds the model architecture definitions (including text and vision components, multi-modal projectors, and attention mechanisms) and the corresponding weight loaders to map HuggingFace PyTorch/safetensors formats to MLC's internal representation. Users can now load and run Gemma3 and Gemma3V models using MLC LLM.
_python/mlc\llm/model/gemma3 · high confidence
Add InternLM model support with multi-GPU tensor parallelism
Users can now run the InternLM language model using MLC LLM. This change introduces the model architecture implementation, including support for tensor parallelism across multiple GPUs to split attention and MLP layers, and utilizes a PagedKVCache for efficient memory management. It also includes a standard loader to map HuggingFace PyTorch or safetensor weights into the MLC format, handling specific weight configurations like the packed QKV projection.
_python/mlc\_llm/model/baichuan, python/mlc\_llm/model/internlm, python/mlc\_llm/model/stable\lm · high confidence
Add LLaVa multimodal model support
Introduces the LLaVa model implementation, including the model architecture (LLaVaForCausalLM), configuration (LlavaConfig), and a parameter loader (llava\_loader.py) that supports conversion from HuggingFace formats and AWQ quantization. This enables users to load and run LLaVa models within the MLC LLM framework.
_python/mlc\llm/model/llava · high confidence
Add Llama2, Cohere Aya-23, and GPTBigCode model implementations
New model architectures are now supported: Llama2 (including Llama3 RoPE scaling), Cohere's Aya-23, and GPTBigCode. Each model includes a dedicated configuration class, a neural network implementation using the Relax frontend, and a parameter loader that maps weights from HuggingFace formats (PyTorch/safetensors) and supports AWQ quantization. The implementations also include tensor-parallel sharding strategies to enable multi-GPU inference.
_python/mlc\llm/model/llama · high confidence
Add MLC WASM runtime library pack
A new source file, mlc\_wasm\_runtime.cc, has been added to the web/emcc directory to serve as the MLC WASM runtime library pack. This file configures the TVM logging stack (disabling stack traces and debug output while enabling custom logging) and defines the COMPILE\_MLC\_WASM\_RUNTIME flag to ensure unsupported code is excluded from the compiled binary.
web/emcc · high confidence
Add MLC-LLM WebAssembly runtime build infrastructure
Users can now build the MLC-LLM WebAssembly runtime directly within the web directory. This change introduces a Makefile and a setup script (prep\_emcc\_deps.sh) that configure the Emscripten compiler to compile the C++ runtime source into a .wasm file. This runtime is designed to be bundled with model libraries, allowing web-based runtimes like WebLLM to reuse the MLC-LLM source code at runtime.
web · high confidence
Add Medusa speculative decoding model support
Users can now load and run Medusa models, which enable speculative decoding to accelerate inference. This change introduces the Medusa model architecture (including residual blocks and multiple heads) and a standard HuggingFace loader that maps parameters from HuggingFace PyTorch or safetensors formats into MLC's internal representation.
_python/mlc\llm/model/medusa · high confidence
Add MiniCPM model support with MoE and weight tying
Users can now load and run the MiniCPM model architecture in MLC LLM. This change introduces the model definition, including support for Mixture-of-Experts (MoE) structures via the MixtralExperts layer, and allows for tied word embeddings between the embedding layer and the language model head. It also provides a parameter loader that correctly maps HuggingFace PyTorch/Safetensor weights (including concatenated QKV and gate/up projections) into the MLC format.
_python/mlc\llm/model/minicpm · high confidence
Add Mixtral model support with MoE and HuggingFace loader
Users can now load and run the Mixtral Mixture-of-Experts (MoE) model. This change introduces the model architecture implementation (including the MoE layers and decoder blocks) and a parameter loader that maps HuggingFace PyTorch/Safetensors weights into MLC's internal format, handling the specific weight concatenation and stacking required for Mixtral's expert routing and attention mechanisms.
_python/mlc\llm/model/mixtral · high confidence
Add Node.js and Python REST API access examples
New example scripts are provided for interacting with the MLC LLM REST API using Node.js and Python. The Node.js examples demonstrate direct HTTP requests, OpenAI-compatible API usage (including streaming), and integration with LangChain for conversational chains and question-answering. The Python examples mirror these capabilities, offering direct requests, OpenAI client usage, and LangChain integrations including retrieval-augmented generation (RAG) with Chroma vector stores. These examples require Node.js v18.17.x and Python with LangChain, serving as reference implementations for client-side integration.
examples/rest · high confidence
Add OLMo2 model support
Users can now run the OLMo2 architecture in MLC LLM. This change introduces the model definition, configuration, and HuggingFace loader in the model module, enabling the use of OLMo2-based models.
_python/mlc\llm/model · high confidence
Add Qwen3 MoE model support
Users can now load and run the Qwen3 Mixture-of-Experts (MoE) language model. This change introduces the model architecture implementation (including MoE-specific layers like sparse MoE blocks and gating mechanisms) and a parameter loader that maps HuggingFace PyTorch/safetensor weights to MLC's internal format, including support for FP8 block-scale quantization.
(repo-wide) · high confidence
Add Qwen3 model support with embedding and FP8 capabilities
Users can now load and run Qwen3 models in MLC LLM. This change introduces the Qwen3 model architecture implementation, including support for Qwen3-Embedding models (via a dedicated loader that handles the absence of the 'model.' prefix) and FP8 block quantization. The model definition has been migrated to use the \tirx\ namespace, and the implementation includes specific handling for Qwen3's attention mechanisms (including query/key normalization) and MLP structures.
_python/mlc\llm/model/qwen3 · high confidence
Add Qwen3.5 text and vision-language model support
Users can now load and run Qwen3.5 models in MLC LLM. This change introduces the Qwen3.5 text model, which uses a hybrid GatedDeltaNet architecture, along with a Qwen3.5 vision-language variant that includes a ViT-based vision encoder with 2D RoPE. The update provides the necessary model definitions, configuration classes, and weight loaders to map Hugging Face checkpoints to the MLC format, including handling fused attention projections and specific normalization offsets.
_python/mlc\llm/model/qwen35 · high confidence
Add RWKV5 model loader for HuggingFace format conversion
The RWKV5 model now includes a dedicated loader module that enables importing weights from HuggingFace PyTorch and safetensor formats. This loader handles specific parameter mappings, including the conversion of time\_decay parameters and the application of rescaling factors for feed-forward and attention output weights, ensuring compatibility with MLC LLM's internal representation.
_python/mlc\llm/model/rwkv5 · high confidence
Add hero section to the website
The website now includes a new hero section on the landing page, featuring the title "MLC LLM: Universal LLM Deployment Engine With ML Compilation," a link to the GitHub repository, and a "Get Started" call-to-action. This section also incorporates a project workflow diagram and includes hover effects for the links.
_site/\includes · high confidence
Add hero section with GitHub link and responsive styling
A new hero section has been added to the website, featuring a radial gradient background, a main heading, and a link container with a GitHub link button and a 'Get Started' button. The section includes a demo container for visual content and implements responsive design adjustments for small screens, including enlarged hero images and adjusted padding/font sizes at various breakpoints (640px, 768px, 1024px).
site/assets · high confidence
Add minimal iOS MLCEngine example app
A new minimal iOS example application has been added to demonstrate interaction with the MLC Engine. The app, built with SwiftUI, loads a packaged Llama-3-8B-Instruct model and performs a chat completion request using the OpenAI-style async API, streaming the response text and usage statistics to the user interface. It also includes necessary entitlements for extended virtual addressing and increased memory limits to support model execution.
ios/MLCEngineExample/MLCEngineExample · high confidence
Add support for Gemma and Gemma 2 models
Users can now load and run Google's Gemma and Gemma 2 large language models. This change introduces the model architectures, configuration handling, and HuggingFace weight loaders for both Gemma (including 1.1 compatibility) and Gemma 2. Gemma 2 support includes specific handling for its sliding window attention and final logit soft-capping, while the Gemma implementation uses a tanh approximation for the GeLU activation in the MLP.
_python/mlc\llm/model/gemma · high confidence
Add support for Microsoft Phi-1, Phi-1.5, and Phi-2 models
Users can now load and run Microsoft's Phi family of models (Phi-1, Phi-1.5, and Phi-2) using MLC LLM. This change introduces the model architecture implementation, configuration classes (Phi1Config, PhiConfig), and parameter loaders that map weights from HuggingFace PyTorch and safetensors formats into MLC's internal representation, enabling inference for these specific model variants.
_python/mlc\llm/model/phi · high confidence
Add support for Microsoft Phi-3 language models
Users can now load and run Microsoft's Phi-3 models within MLC LLM. This change introduces the Phi-3 model architecture, including its configuration, parameter loading from HuggingFace formats, and tensor-parallel execution support. It also handles specific Phi-3 features such as long-rope scaling and configurable prefill chunk sizes.
_python/mlc\llm/model/phi3 · high confidence
Add support for the InternLM2 model architecture
Users can now load and run the InternLM2 large language model. This change introduces the model definition, configuration, and a parameter loader that maps HuggingFace weights (PyTorch/safetensors) to MLC's internal format, including handling of the combined QKV attention weights and gated MLP structures. It also includes support for tensor parallelism across multiple GPUs.
(repo-wide) · high confidence
Add support for the Orion-14B model architecture
Users can now load and run the Orion-14B model using MLC LLM. This change introduces the model definition, configuration handling, and a HuggingFace weight loader for Orion, enabling inference with this specific architecture.
_python/mlc\llm/model/orion · high confidence
Centralized quantization registry and FP8 support for Mixtral experts
The quantization module now uses a centralized registry (QUANTIZATION) and factory functions to manage quantization configurations, making it easier to select and apply different quantization schemes. This change introduces native FP8 per-tensor quantization support for Mixtral experts, including calibration modes (max and inference) and optimized matmul operations. Additionally, it adds block-scale quantization for FP8 with bfloat16 models and standardizes the quantization API across different model types.
_python/mlc\llm/quantization · high confidence
Experimental embeddings API added to contrib
A new experimental embeddings module has been added to the \mlc\_llm.contrib\ package, providing a Python API for generating embeddings using MLC LLM encoder models. This includes a core \MLCEmbeddings\ class that loads compiled model libraries and executes inference via the TVM Virtual Machine, along with a LangChain-compatible adapter (\mlc\_llm.contrib.embeddings.openai\) that implements the \OpenAIEmbeddings\ interface for seamless integration with existing LangChain workflows.
_python/mlc\llm/contrib · high confidence
Initial implementation of the Mistral model architecture and loader
This change introduces the core implementation for the Mistral language model within MLC LLM. It adds the \MistralConfig\ class to handle model configuration parameters such as sliding window size, prefill chunk size, and tensor parallelism settings, including automatic fallbacks for context window detection. The \MistralForCausalLM\ model definition implements the decoder layers, attention mechanisms using \PagedKVCache\, and MLP structures compatible with tensor parallel execution. Additionally, a new \mistral\_loader.py\ provides parameter mapping logic to convert weights from HuggingFace formats (PyTorch/safetensors) and supports AWQ quantization by handling the concatenation of QKV and gate/up projections.
_python/mlc\llm/model/mistral · high confidence
Initial release of the MLCChat Android application
This change introduces the MLCChat Android app, providing a native interface for running local large language models on Android devices. The app features a chat interface with Markdown rendering for assistant responses, model management for downloading and selecting models, and support for image inputs via camera or gallery. It is built using Jetpack Compose and Material Design 3, and includes a configuration file (\mlc-package-config.json\) pre-configured with a list of supported models such as Phi-3.5, Qwen3, Gemma-2, Llama-3.2, and Mistral-7B.
android/MLCChat · high confidence
Initial release of the mlc\_llm Python package
The \python/mlc\llm\ directory is introduced as a new Python package, establishing the core runtime and CLI infrastructure for MLC LLM. This includes the main entry point (\\\main\\.py\) exposing subcommands such as \compile\, \chat\, \serve\, and \calibrate\, alongside foundational modules for library loading (\base.py\, \libinfo.py\) and engine exports (\\\init\\_.py\).
_python/mlc\llm · high confidence
Initial support for Ministral 3 model architecture
Users can now load and run the Ministral 3 model in MLC LLM. This change introduces the model definition, configuration, and a parameter loader that supports mapping from HuggingFace checkpoints. It specifically enables FP8 block-scale quantization, handling the reconstruction of weights and scales from block-quantized storage, and includes logic to manage activation scales and modules that should not be converted during quantization.
_python/mlc\llm/model/ministral3 · high confidence
Initial support for the Qwen2 model architecture
Users can now load and run Qwen2 models using MLC LLM. This change introduces the core model implementation (including attention, MLP, and embedding layers), a configuration class with defaults for prefill chunk size and batch size, and a HuggingFace loader to map weights from standard formats. It also enables tensor-parallel sharding for multi-GPU inference and supports weight tying between the embedding and language model head.
_python/mlc\llm/model/qwen2 · high confidence
Interactive CMake configuration script for build backends
The build system now includes a Python script (cmake/gen\_cmake\_config.py) that interactively prompts users to select enabled hardware backends (CUDA, OpenCL, ROCm, Vulkan, Metal, CUTLASS, CUBLAS, and OpenCL Host Ptr) and generates a config.cmake file. The script automatically enables Thrust when CUDA is selected and manages parent-child backend dependencies, simplifying the initial setup process for developers.
cmake · high confidence
Introduce C++ serving engine with structured configuration and data abstractions
The serving engine now uses explicit C++ configuration and data structures for request handling. EngineConfig and GenerationConfig are parsed from JSON, supporting modes (local, interactive, server) and generation parameters (temperature, top\_p, penalties, logprobs, response format). DebugConfig allows controlling grammar execution (jump-forward or constraint), ignoring EOS, and handling disaggregation requests (prepare\_receive, remote\_send, start\_generation) with KV window metadata. Data abstractions (TextData, TokenData, ImageData) unify input handling, and a DraftTokenWorkspaceManager allocates GPU workspace for speculative decoding draft tokens. The engine state tracks running/waiting queues, prefix cache, and metrics, with a mock echo engine for testing.
cpp/serve · high confidence
Introduce EAGLE speculative decoding model support
Adds the EAGLE model implementation and its HuggingFace/AWQ parameter loader to MLC LLM, enabling users to utilize EAGLE for speculative decoding. The new module includes the model architecture (EagleForCausalLM) which extends Llama components with specific fusion layers, and a loader that handles weight mapping from standard formats and AWQ quantization, allowing these models to be loaded and run within the existing MLC infrastructure.
_python/mlc\llm/model/eagle · high confidence
Introduce JSON FFI Engine for Android LLM inference
The mlc4j module now includes a new JSONFFIEngine class that exposes LLM capabilities to Android applications via a JSON-based Foreign Function Interface. This engine initializes a background inference process, manages model reloading and unloading, and handles chat completions by streaming responses through a callback mechanism, allowing apps to interact with the underlying TVM runtime without direct C++ bindings.
android/mlc4j/src/main · high confidence
Introduce JSON FFI Engine with vision and function-calling support
The cpp/json\_ffi directory now contains the initial implementation of the JSONFFIEngine, providing a new C++ interface for serving LLMs via JSON-based requests. This engine supports multimodal inputs by including image utility functions to load and preprocess base64-encoded images into tensors for CLIP encoders. It also implements the OpenAI API protocol, enabling function calling through structured tool definitions and message handling. Additionally, the new conv\_template module allows for customizable conversation templates with support for system prompts, role-based text generation, and stop criteria, making the engine adaptable to various model architectures.
_cpp/json\ffi · high confidence
Introduce JSON FFI engine for string-based API interaction
A new JSON FFI engine has been added to the Python API, providing a pure string-based interface for interacting with the MLC LLM Engine. This component, exposed via \python/mlc\_llm/json\_ffi\, is designed primarily for testing purposes and internal use, offering an alternative to the standard \MLCEngine\ and \MLCAsyncEngine\ interfaces. The implementation includes an \EngineState\ class to manage request streams and a \Completions\ class that adheres to the OpenAI API protocol, allowing users to send chat completion requests and receive streaming responses in JSON format.
_python/mlc\_llm/json\ffi · high confidence
Introduce MLC LLM Python package with CLI entrypoint
The Python package is now installable via pip as \mlc\_llm\, providing a formal structure for the library. This change adds a command-line interface accessible via the \mlc\_llm\ console script, allowing users to interact with the tool directly from the terminal. The package configuration handles binary distribution requirements and manages dependencies defined in \requirements.txt\.
python · high confidence
Introduce MLC LLM benchmarking tool
Adds a new benchmarking suite for MLC LLM, allowing users to evaluate model performance against various API backends (including OpenAI-compatible endpoints) and datasets. The tool supports configurable request rates, concurrent request counts, and specific engine configurations, outputting detailed metrics and CSV reports for analysis.
_python/mlc\llm/bench · high confidence
Introduce MLCEngine and OpenAI-compatible API for Android
The mlc4j Android library now provides a new MLCEngine class that wraps the JSON FFI engine to handle background model execution and streaming responses. This change introduces a high-level API compatible with the OpenAI v1/chat/completions protocol, including support for chat messages, tools, and streaming usage statistics, replacing the previous legacy ChatModule approach.
mlc4j · high confidence
Introduce MLCEngine with OpenAI-compatible Swift API
The iOS Swift SDK now provides a new \MLCEngine\ class that exposes an API aligned with the OpenAI Chat Completions standard. This includes support for structured request inputs (messages, tools, logprobs, etc.), streaming responses via \AsyncStream\, and background execution to handle engine callbacks. The implementation bridges to the underlying JSON FFI engine, allowing iOS developers to integrate LLM capabilities using familiar OpenAI-style method signatures.
ios/MLCSwift/Sources/Swift · high confidence
Introduce Microsoft Phi-3 Vision model support
Adds the Microsoft Phi-3 Vision model to MLC LLM, enabling users to run this multimodal model that processes both text and images. The implementation includes the core model architecture (Phi3VForCausalLM), a vision embedding component (Phi3ImageEmbedding) that integrates a CLIP vision encoder with image projection layers, and a parameter loader (phi3v\_loader) to map HuggingFace PyTorch weights to MLC's internal format. The model supports dynamic shapes, RoPE scaling (longrope), and specific optimizations like WebGPU uint32 image embedding dtypes.
_python/mlc\llm/model/phi3v · high confidence
Introduce Objective-C wrapper for JSON FFI LLM engine on iOS
Added an Objective-C interface (LLMEngine) that exposes the underlying JSON FFI engine to Swift, enabling iOS apps to initialize the background engine, reload configurations, reset state, and manage chat completions via JSON requests.
ios/MLCSwift/Sources/ObjC · high confidence
Introduce Qwen2.5-VL model entry point
The Qwen2.5-VL model is now available in the library. This change adds the model's entry point, exposing the configuration class (Qwen25VLConfig) and the language model head (Qwen25VLLMHeadModel) for use.
_python/mlc\_llm/model/qwen2\_5\vl · high confidence
Introduce centralized HuggingFace model weight loader
The \mlc\_llm.loader\ package now provides a unified system for loading model weights from HuggingFace repositories. It introduces \HuggingFaceLoader\ to handle PyTorch and SafeTensor formats, including support for sharded weights and quantization mappings. A centralized registry (\LOADER\) exposes loaders for \huggingface-torch\, \huggingface-safetensor\, and \awq\ formats. Additionally, \make\_standard\_hf\_loader\ offers a helper to generate standard parameter mappings for models with common architectures (e.g., QKV/gate-up concatenation), simplifying the integration of new HuggingFace models.
_python/mlc\llm/loader · high confidence
Introduce centralized support utilities for MLC LLM
This change introduces a new \python/mlc\_llm/support\ package that consolidates common utilities previously scattered across the codebase. It provides enhanced argument parsing with improved error reporting, automatic detection of local devices (GPU/CPU) and compilation targets, and robust configuration handling for models, weights, and quantization schemes. The package also standardizes download caching from HuggingFace with configurable policies, adds support for converting tiktoken tokenizers to HuggingFace format, implements weight pre-sharding for tensor parallelism, and includes shared logging, styling, and random seed management utilities.
_python/mlc\llm/support · high confidence
Introduce dedicated CPU and GPU samplers for text generation
The serving layer now uses separate CPU and GPU sampler implementations to handle token generation. The new CPU sampler provides optimized top-p sampling with early-exit logic and pivot-based renormalization, while the GPU sampler leverages FlashInfer for accelerated sampling on CUDA devices and supports Metal and Vulkan backends. This change introduces a unified \Sampler\ interface that allows the system to select the appropriate backend based on the available hardware, improving performance and enabling efficient speculative decoding verification on the GPU.
cpp/serve/sampler · high confidence
Introduce dedicated Python interface modules for MLC LLM operations
The \python/mlc\_llm/interface\ package has been restructured into distinct entrypoint modules (\calibrate.py\, \gen\_config.py\, \help.py\, \package.py\, \router.py\, \serve.py\) to separate concerns for model calibration, configuration generation, CLI help, mobile packaging, request routing, and model serving. This change provides a cleaner, modular API surface for users and developers interacting with MLC LLM's core capabilities, replacing the previous monolithic or less-organized structure with specific, focused modules for each major workflow.
_python/mlc\llm/interface · high confidence
Introduce initial CI infrastructure for building and testing MLC LLM
This change adds the foundational CI pipeline for the project, introducing a Jenkins-based workflow that builds and packages MLC LLM runtime libraries for CUDA, Metal, and Vulkan targets, and runs unit tests and model compilation checks. It includes a new \ci/bash.sh\ script to standardize Docker container execution (handling GPU passthrough, environment variables, and volume mounts), a \ci/build-environment.yaml\ defining the conda build environment (pinning Python 3.11 and specific Vulkan/SPR tools), and a \jenkinsfile.groovy\ orchestrating parallel builds and tests across different hardware backends.
ci · high confidence
Introduce mlc4j build system for Android library packaging
Adds a new build infrastructure for the Android MLC LLM library (mlc4j), including a CMakeLists.txt configuration, a Python preparation script (prepare\_libs.py), and a .gitignore file. This setup automates the compilation of the TVM4J runtime and MLC LLM static libraries for Android (arm64-v8a), handling JNI header generation, dependency linking, and installation of the resulting artifacts for app packaging.
android/mlc4j · high confidence
Introduce modular neural network components for MoE, KV caching, and RNN states
The \python/mlc\llm/nn\ package now provides core neural network building blocks. It exports \MixtralExperts\ for handling Mixture-of-Experts attention with optimized GEMM backends (Cutlass, FasterTransformer, or standard GEMM), a \PagedKVCache\ class that wraps TVM's paged cache implementation to support various attention modes (MHA, MLA, sliding window) and configuration options, and an \RNNState\ class for managing state in State Space Models via dedicated get/set operations. These modules are exposed via the \\\init\\_.py\ entry point.
_python/mlc\llm/nn · high confidence
Introduce multi-GPU model loading with pre-sharding support
Added new C++ implementation files (\builtin.cc\ and \multi\_gpu\_loader.cc\) in the \cpp/multi\_gpu\ directory to enable multi-GPU inference with loading-time parameter sharding. The \multi\_gpu\_loader.cc\ file implements the \LoadMultiGPU\ function, which handles model metadata extraction, parameter loading, preprocessing, and scattering across GPU groups, while \builtin.cc\ registers helper functions like \DispatchFunctionByGroup\ and \SendFromLastGroupToWorker0\ for inter-worker communication and execution dispatch.
_cpp/multi\gpu · high confidence
Introduce programmable router for microserving endpoints
Added a new \Router\ component in \python/mlc\_llm/router\ that dispatches OpenAI API requests to multiple underlying engine endpoints. The router supports two modes: 'disagg' (disaggregated) and 'round-robin' scheduling, allowing users to route traffic across multiple server instances. It handles request translation, endpoint selection based on congestion (for disagg mode) or round-robin, and manages the lifecycle of underlying \PopenServer\ instances, including NVSHMEM initialization for distributed GPU communication.
_python/mlc\llm/router · high confidence
Introduce structured protocol definitions for MLC LLM configuration and serving
This change establishes a new \python/mlc\_llm/protocol\ module that defines Pydantic models for the library's core data structures. It introduces \MLCChatConfig\ to manage model settings (including \active\_vocab\_size\ and \model\_task\), \GenerationConfig\ for sampling parameters, and \DebugConfig\ for engine-level options like jump-forward decoding. Additionally, it adds specific protocols for MicroServing (handling KV cache windowing and metadata) and standardized error responses, providing a consistent, validated interface for API entry points and internal configuration.
_python/mlc\llm/protocol · high confidence
Introduces compiler pass library for GPU sampling, logit processing, and CUDA graph support
The \python/mlc\_llm/compiler\_pass\ module is introduced, providing a suite of compiler passes that attach optimized GPU kernels to the model's IRModule. This includes GPU sampling functions (multinomial, top-p, argsort) for CUDA, Vulkan, Metal, and WebGPU targets, as well as logit processing kernels for bias, penalties, and bitmasks. The module also adds passes for two-stage softmax with temperature, CUDA graph allocation initialization, and dispatching to cuBLAS/hipBLAS, enabling more efficient inference on supported hardware.
_python/mlc\_llm/compiler\pass · high confidence
Introduction of AppViewModel for model and chat state management
The app now uses a new \AppViewModel\ class to manage the application's state, including the list of available models, chat history, and configuration. This component handles loading app settings from local storage or assets, downloading model configurations, and managing model deletion. It also provides mechanisms for displaying alerts and copying error messages to the clipboard, serving as the central logic layer for the user interface's data needs.
android/MLCChat/app · high confidence
New C++ tokenizer and streaming components
Added C++ implementations for the tokenizer wrapper (tokenizers.cc/h) and streaming utilities (streamer.cc/h). The tokenizer now supports loading from HuggingFace tokenizer.json, SentencePiece tokenizer.model, and ByteLevelBPE files, with automatic detection and priority for HuggingFace/SentencePiece formats. The TextStreamer component ensures UTF-8 validity when streaming decoded text by handling partial multi-byte characters, while the StopStrHandler uses the KMP algorithm to efficiently detect stop strings during generation.
cpp/tokenizers · high confidence
New CLI entrypoints for calibration, device checking, and model metadata inspection
The \python/mlc\_llm/cli\ package now includes dedicated command-line entrypoints for several new capabilities. Users can run \mlc\_llm calibrate\ to perform FP8 calibration using a specified dataset and model, and \mlc\_llm check\_device\ to verify the availability of specific hardware devices. Additionally, a new \mlc\_llm model\_metadata\ tool allows users to inspect compiled model libraries, reporting detailed memory usage (parameters and temporary buffers) and KV cache requirements. The package also introduces \disco\_remote\_socket\_session\ and \worker\ scripts to support distributed Disco process sessions.
_python/mlc\llm/cli · high confidence
New CLI tools for model delivery, library compilation, and serving
The \mlc\_llm\ package now includes dedicated command-line interfaces for managing model artifacts and running inference. The \delivery\ CLI automates downloading models from Hugging Face, generating configuration, converting weights, and uploading the resulting packages. The \lib\_delivery\ CLI handles continuous compilation of model libraries for various devices and quantizations based on a JSON specification. Additionally, the \serve\ CLI provides a direct entry point to start an inference server with configurable engine parameters, while the \chat\ CLI offers an interactive text-based chat interface. These tools streamline the workflow from model preparation to deployment. Also in this area: 3 commits (2 fixes) fixing python.
_python/mlc\llm/model/olmo2 · medium confidence · unverified
New MLCEngineExample Android app using JSON FFI
A new Android example application (MLCEngineExample) is introduced, demonstrating the JSON FFI Engine interface. The app uses Jetpack Compose and Material 3 to display streaming LLM responses, specifically loading the phi-2-q4f16\_1-MLC model via the MLCEngine API. It includes a Python utility (bundle\_weight.py) to assist in pushing model weights to the device and is configured via mlc-package-config.json for the Android target.
android/MLCEngineExample · high confidence
New Python examples for MLC Engine and Microserving
Added two new Python example scripts: \sample\_mlc\_engine.py\ demonstrates how to initialize and use the \MLCEngine\ for chat completions via the OpenAI API, and \custom\_router.py\ provides a template for implementing a custom router in the Microserving architecture, showing how to handle request translation and coordinate between prefill and decode engines.
examples/python · high confidence
New Python serving API with embedding support and paged prefix caching
The \python/mlc\_llm/serve\ package now exposes a unified Python API for LLM and embedding inference. Users can initialize \AsyncMLCEngine\ or \MLCEngine\ via \EngineConfig\ to run text generation with features like speculative decoding (Eagle, Medusa), paged radix-tree prefix caching, and hybrid prefill modes. The package also introduces \AsyncEmbeddingEngine\ for encoder and decoder embedding models, accessible via the new \mlc\_llm.serve\ namespace.
_python/mlc\llm/serve · high confidence
New conversation templates for Llama 4, Ministral 3, Qwen 3.5, and DeepSeek-V3
This change adds conversation templates for several new and updated models, ensuring correct formatting and stop tokens for Llama 4, Ministral 3 (including a reasoning variant), Qwen 3.5 (with reasoning block stripping), and DeepSeek-V3 (including R1 distill variants). It also introduces templates for Hermes 3 (Llama 3.1), Gemma 4, and LLM-jp, while updating existing templates for Llama 3.1 and Qwen 2/3 to match their latest specifications.
_python/mlc\_llm/conversation\template · high confidence
New external operator library for attention, MoE, and quantized GEMM
The \mlc\_llm.op\ package now exposes a comprehensive set of external operators for high-performance inference. This includes a FlashInfer-backed attention kernel with configurable scaling and fallback paths, Mixture-of-Experts (MoE) operators for gating, top-k selection, and quantized GEMV, and specialized FP8 GEMM implementations via CUTLASS and Triton. Additionally, the library introduces operators for batch speculative decoding verification, top-p sampling pivots, and pipeline parallelism stage boundaries.
_python/mlc\llm/op · high confidence
New server entrypoints module with OpenAI, MicroServing, debug, and metrics APIs
The \python/mlc\_llm/serve/entrypoints\ package has been introduced to organize the server's HTTP API routes. This module exposes OpenAI-compatible endpoints (\/v1/completions\, \/v1/embeddings\, \/v1/models\) with support for API key authentication, embedding dimension truncation, and streaming responses. It also adds MicroServing endpoints (\/microserving/prep\_recv\, \/microserving/remote\_send\, \/microserving/start\_generate\) for disaggregated serving workflows, debug endpoints (\/debug/dump\_event\_trace\, \/debug/cuda\_profiler\_start/stop\, \/debug/reset\_engine\) for operational control, and a Prometheus metrics endpoint (\/metrics\).
_python/mlc\llm/serve/entrypoints · high confidence
New server subprocess management and global context utilities
The \python/mlc\_llm/serve/server\ package now provides \PopenServer\ and \ServerContext\ classes to manage the LLM serving lifecycle. \PopenServer\ allows launching the MLC LLM server in a background subprocess with configurable host, port, and engine parameters, handling startup readiness checks and clean termination of child processes. \ServerContext\ offers a global singleton to track running model engines and embedding engines, facilitating multi-model serving and resource cleanup via context managers.
_python/mlc\llm/serve/server · high confidence
New support utilities for debugging, encoding, and vision models
The \cpp/support\ directory now includes a suite of new utility headers and source files to enhance development and model handling. A \DebugRegistry\ allows registering and retrieving tokenizers for easier debugging. New \encoding\ utilities provide robust UTF-8 parsing, codepoint printing, and escape sequence handling. A \DynamicBitset\ class supports runtime-sized bit manipulation for grammar-guided generation, while a \Result\ class standardizes error handling across the codebase. JSON parsing is now handled via a dedicated \json\_parser\ with typed lookup helpers. Additional utilities include a \ProgressBar\ for status updates, a \RandomGenerator\, file loading helpers, and a \module\_vtable\ compatibility shim for TVM runtime changes. Finally, \vlm\_utils\ provides image preprocessing logic (resize, pad, crop) specifically for Phi-3, Gemma 3, and Qwen 3.5 vision models.
cpp/support · high confidence
Removals
Removal of legacy C++ CLI and ChatModule
The legacy C++ command-line interface (cli\_main.cc) and the core LLM chat implementation (llm\_chat.cc) have been removed from the codebase. This change eliminates the previous C++-based chat module and its associated conversation templates, aligning with the project's shift toward Python-based CLI tools and the new SLM architecture. The header file llm\_chat.h has been renamed to base.h, stripping out the C++ chat module creation API.
cpp · high confidence
Removed custom C++ tokenizer wrapper
The custom C++ wrapper for the tokenizer library, including its C bindings and CMake build configuration, has been removed from the 3rdparty directory. This eliminates the local Rust-based tokenizer implementation and its associated build logic, indicating a shift to an alternative tokenization approach elsewhere in the codebase.
3rdparty/tokenizers-cpp · high confidence
Behavioural changes
Adopts scikit-build-core for Python packaging and introduces pre-commit linting
The project has migrated its Python build system from the legacy setup.py to scikit-build-core, enabling modern wheel building and aligning with the CMake-based build infrastructure. To support this transition, the old build.py and setup.py scripts have been removed. Additionally, a pre-commit configuration has been added to enforce code quality using ruff for Python linting/formatting, clang-format for C/C++, and yamllint/taplo for configuration files, ensuring consistent code style across the repository.
(repo-wide) · high confidence
Android TVM runtime logging redirected to Android logcat
The MLC4J Android runtime now routes TVM log messages directly to the Android logcat system. This is achieved by overriding the internal logging implementation in the C++ runtime layer to use \\_\_android\_log\_write\, ensuring that debug and fatal logs from the ML inference engine are visible in standard Android device logs.
android/mlc4j/src/cpp · high confidence
Introduce structured C++ model metadata abstraction
The \cpp/metadata\ location now provides a dedicated C++ abstraction for model metadata, replacing previous ad-hoc handling with a structured \ModelMetadata\ class. This new component parses model configuration from JSON and TVM modules, exposing specific fields such as \max\_batch\_size\, \context\_window\_size\, \sliding\_window\_size\, \attention\_sink\_size\, and \kv\_state\_kind\ (supporting \kv\_cache\, \rnn\_state\, \none\, and \hybrid\ states). It also captures parameter details including dynamic shapes, preprocessing functions, pipeline parallel stages, and embedding metadata, enabling the runtime to accurately interpret model capabilities and memory requirements.
cpp/metadata · high confidence
MLC Chat iOS app restructured with updated model configuration
The ios/MLCChat directory has been restructured, removing legacy Swift and Objective-C implementation files (ChatState, ChatView, LLMChat.mm, etc.) and introducing a new README and mlc-package-config.json. The configuration file now defines a model list including Llama-3.2-3B-Instruct, Gemma-2-2B, Phi-3.5-mini, and Qwen3 variants, specifying their Hugging Face sources, estimated VRAM usage, and inference overrides like prefill chunk size and context window size.
ios/MLCChat · high confidence
New CI task scripts for building and testing MLC LLM
Added shell and batch scripts in the ci/task directory to standardize the CI workflow. build\_clean.sh removes build artifacts, build\_lib.sh handles library compilation with support for CUDA, ROCm, Metal, and Vulkan backends using ccache and auditwheel, build\_win.bat manages Windows-specific builds by resolving linker conflicts, test\_model\_compile.sh validates model compilation across various GPU targets including WebGPU and iOS, and test\_unittest.sh executes unit tests with necessary dependency installation.
ci/task · high confidence
Refactor MLC LLM CLI and interface entrypoints into modular Python modules
The command-line interface and Python entrypoints for MLC LLM have been restructured into distinct, modular files under the \mlc\_llm/interface\ and \mlc\_llm/cli\ packages. This change introduces dedicated modules for serving (\serve.py\), chat (\chat.py\), compilation (\compile.py\), weight conversion (\convert\_weight.py\), and model delivery (\delivery.py\, \lib\_delivery.py\). It also adds a new benchmarking request record system (\bench/request\_record.py\) for detailed performance metrics and introduces an \EngineConfigOverride\ mechanism in the serve CLI to allow runtime configuration overrides (e.g., for prefill mode, prefix cache, and speculative decoding). Users will now interact with a more organized CLI structure that supports finer-grained control over engine behavior, compilation flags, and model delivery workflows.
_mlc\llm · high confidence
Refactor documentation build and deployment scripts
The documentation build process has been restructured to separate concerns: new scripts \build\_mlc\_for\_docs.sh\ and \build\_site.sh\ now handle the CMake/Make build of MLC components and the Sphinx/Jekyll site generation respectively, replacing the previous monolithic approach. The \prep\_deps.sh\ script has been removed, and \gh\_deploy\_site.sh\ and \local\_deploy\_site.sh\ have been updated to invoke these new build scripts. Additionally, a new \check\_url\_validity.py\ script was added to validate links in documentation files.
scripts · high confidence
Refactored engine execution into modular action classes
The engine's core execution loop has been restructured to use a modular \EngineAction\ abstraction. The \cpp/serve/engine\_actions\ directory now contains distinct action classes for specific engine steps, including \NewRequestPrefill\, \BatchDecode\, \BatchDraft\, \BatchVerify\, and \BatchJumpForward\. A new \CreateEngineActions\ factory function in \action\_commons\ dynamically assembles the correct sequence of these actions based on the \EngineConfig\ (e.g., selecting between standard, Eagle, Medusa, or automatic speculative decoding modes). This change replaces the previous monolithic step logic with a composable pipeline of specialized actions.
_cpp/serve/engine\actions · high confidence
Restructure iOS app with new state management and model configuration
The iOS app has been restructured to introduce a new \AppState\ and \ModelState\ for managing model downloads and chat sessions, replacing the previous flat \ChatState\ as the primary entry point. This change adds support for loading app and model configurations from JSON files (\mlc-app-config.json\, \mlc-chat-config.json\, \tensor-cache.json\) and enables downloading models from remote URLs. The UI now presents a \StartView\ for managing multiple models, each with its own download progress and deletion options, rather than immediately launching a chat. Additionally, the app entitlements have been updated to include \com.apple.developer.kernel.extended-virtual-addressing\ to support larger memory usage.
ios/MLCChat/MLCChat · high confidence
Restructure iOS build system and add Mac Catalyst support
The iOS build process has been reorganized to support Mac Catalyst (macabi) and iOS Simulator architectures (x86\_64, arm64) via a revamped \prepare\_libs.sh\ script. The project structure was updated to include a new \MLCEngineExample\ Xcode project and reorganized \MLCChat\ workspace files, while the legacy \prepare\_params.sh\ script was removed.
ios · high confidence
Website rebranding to llm.mlc.ai with updated homepage and navigation
The project website has been migrated from mlc.ai/mlc-llm to the new domain llm.mlc.ai, updating the site configuration, CNAME, and base paths. The homepage content has been significantly simplified to focus on an overview and direct links to the new documentation site, replacing the previous detailed local installation instructions. A new privacy policy page has been added, and the main navigation menu now includes a link to the Docs section.
site · high confidence
iOS app project restructured with new UI components and dependencies
The iOS project has been reorganized, moving the project file to a nested directory and updating the Xcode workspace version. This change introduces several new Swift source files including StartView, ModelView, AppState, ModelConfig, ModelState, ImageProcessing, ParamsConfig, AppConfig, and Constants, while removing older files like ThreadWorker and LLMChat.mm. The app now embeds the MLCSwift and MarkdownUI frameworks, enabling markdown text display in the chat interface, and updates the copy files phase to include a 'bundle' directory instead of the previous 'dist' folder.
ios/MLCChat/MLCChat.xcodeproj · high confidence
Fixes
3 commits (2 fixes) fixing python
A fix in python — 3 commits (2 fixs), 102 files: “\[Fix\] Correct Gemma 3 sliding-window attention (\#3588)”; “\[Deps\] Bump TVM and adapt to the TIRx split (\#3583)”; “\[Fix\] Remove a shared-memory race in the softmax\_with\_temperature reduction (\#3580)” (refs \#3588, \#3583, \#3580). Also in this area: Add Gemma 4 text and audio model support.
_python/mlc\llm/model/gemma4 · medium confidence · unverified
Test coverage
24 commits adding/updating tests in python/mlc\_llm/testing; Add server integration tests for embedding, function calling, and image capabilities; Added Python tests for JSON FFI Engine capabilities; Added comprehensive test suite for MLC LLM serving engine; Added correctness tests for new speculative decoding and FP8 operations; Added integration test for model compilation across multiple targets and quantizations; Added model architecture tests for Gemma 3/4, Qwen 3.5, and other models; Added test categorization markers for Python test suite; Added tests for AWQ and HuggingFace model loading; Added tests for AWQ and group quantization; Added tests for MLC LLM compiler passes; Added tests for artifact manifest validation and contract consistency; Added tests for conversation templates and tokenizers; Added tests for the Python router with multiple endpoint configurations; Added unit tests for JSONFFI conversation template parsing; Added unit tests for MLC LLM support utilities; Removal of debug comparison library; Reorganize test infrastructure and remove legacy scripts.
Dependencies
57 commits updating dependencies (23 manifests)
A dependency / build maintenance change in (dependencies) — 57 commits (8 fixs), 23 files.
(dependencies) · medium confidence · unverified
Updated 3rd-party submodules and removed sentencepiece-js
The 3rd-party dependencies have been updated to new submodule commits: googletest, stb, tokenizers-cpp, tvm, and xgrammar. Additionally, the sentencepiece-js submodule has been removed from the repository.
3rdparty · high confidence
Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 50 → 54 (+4.1)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.2) — scores are not directly comparable.
Lenses
- Code Health 87 → 87 (-0.0)
- Architecture 97 → 97 (+0.0)
- Maturity 37 → 59 (+21.3)
- Readiness 40 → 40 (+0.0)
- Security 66 → 55 (-10.6)
- Performance 100 → 100 (+0.0)
Resolved (45)
- Duplicated block (10 lines × 2) (python/mlc_llm/op/moe_misc.py)
- Duplicated block (10–11 lines × 2) (python/mlc_llm/nn/rnn_state.py)
- Duplicated block (11 lines × 2) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (11 lines × 23) (python/mlc_llm/model/chatglm3/chatglm3_model.py)
- Duplicated block (12 lines × 2) (python/mlc_llm/compiler_pass/attach_logit_processor.py)
- Duplicated block (13 lines × 2) (python/mlc_llm/compiler_pass/fuse_add_norm.py)
- Duplicated block (23–27 lines × 3) (python/mlc_llm/compiler_pass/fuse_ft_dequantize_matmul_epilogue.py)
- Duplicated block (27–29 lines × 2) (python/mlc_llm/compiler_pass/fuse_ft_dequantize_matmul_epilogue.py)
- Duplicated block (33 lines × 2) (python/mlc_llm/nn/rnn_state.py)
- Duplicated block (4–8 lines × 5) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (5 lines × 2) (python/mlc_llm/compiler_pass/attach_softmax_with_temperature.py)
- Duplicated block (5 lines × 2) (python/mlc_llm/compiler_pass/fuse_add_norm.py)
- Duplicated block (5 lines × 2) (python/mlc_llm/model/rwkv5/rwkv5_model.py)
- Duplicated block (5 lines × 3) (python/mlc_llm/model/qwen35/qwen35_model.py)
- Duplicated block (5 lines × 7) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (6 lines × 2) (python/mlc_llm/model/gemma/gemma_model.py)
- Duplicated block (6 lines × 3) (python/mlc_llm/op/batch_spec_verify.py)
- Duplicated block (6 lines × 5) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (9 lines × 2) (python/mlc_llm/model/rwkv5/rwkv5_model.py)
- Duplicated block (9 lines × 2) (python/mlc_llm/op/batch_spec_verify.py)
- …and 25 more
New (95)
- Concentrated knowledge decay
- Duplicated block (10 lines × 2) (python/mlc_llm/op/moe_misc.py)
- Duplicated block (10–11 lines × 2) (python/mlc_llm/nn/rnn_state.py)
- Duplicated block (11 lines × 2) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (12 lines × 3) (python/mlc_llm/model/gemma3/gemma3_model.py)
- Duplicated block (13 lines × 2) (python/mlc_llm/compiler_pass/fuse_add_norm.py)
- Duplicated block (13 lines × 2) (python/mlc_llm/op/batch_spec_verify.py)
- Duplicated block (16 lines × 2) (python/mlc_llm/compiler_pass/attach_logit_processor.py)
- Duplicated block (23–27 lines × 3) (python/mlc_llm/compiler_pass/fuse_ft_dequantize_matmul_epilogue.py)
- Duplicated block (33 lines × 2) (python/mlc_llm/nn/rnn_state.py)
- Duplicated block (4–8 lines × 5) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (5 lines × 2) (python/mlc_llm/compiler_pass/attach_softmax_with_temperature.py)
- Duplicated block (5 lines × 2) (python/mlc_llm/compiler_pass/fuse_add_norm.py)
- Duplicated block (5 lines × 2) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (5 lines × 3) (python/mlc_llm/model/qwen35/qwen35_model.py)
- Duplicated block (5 lines × 4) (python/mlc_llm/compiler_pass/attach_logit_processor.py)
- Duplicated block (5 lines × 7) (python/mlc_llm/op/moe_matmul.py)
- Duplicated block (9 lines × 2) (python/mlc_llm/model/rwkv5/rwkv5_model.py)
- Hotspot: python/mlc_llm/interface/compile.py (python/mlc_llm/interface/compile.py)
- Members sharing a duplicated core (10 members, 50+ identical tokens) (python/mlc_llm/model/baichuan/baichuan_model.py)
- …and 75 more
Changes since last survey
- 3 commits — 1 feature/other, 2 fixes
By area
- python/mlc_llm — 3 commits
Notable commits
- fix: [Fix] Correct Gemma 3 sliding-window attention (#3588)
- fix: [Fix] Remove a shared-memory race in the softmax_with_temperature reduction (#3580)
- change: [Deps] Bump TVM and adapt to the TIRx split (#3583)
Architecture
- Containers 0 added · 0 removed · contexts 1 added · 1 removed · edges 1 added · 1 removed
Added bounded contexts (1)
- android/MLCChat/app
Removed bounded contexts (1)
- android/MLCChat/app
Added dependency edges (1)
- android/MLCChat/app → mlc4j
Removed dependency edges (was a dependency, now gone) (1)
- android/MLCChat/app → mlc4j
Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
mlc-ai/mlc-llm was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 4 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 44fd06d04f22c87820e205b0f2b2e5e6a1f6ca85 — the exact code this score is about.
- Scored under rubric-2026.11.2 — the same rubric and the same method as every other entry in this index.
- Measured by watchdog.canine.dev using codehealth-analyzer measure-2026.11.2+preprod-8dd06085e9a4.