Skip to content
CAI
Software that uses CAICheck a score

apple/coreai-models

64.2

Adequate · 15 November 2026

85.5k

lines of production code

Swift

with Python

5

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Features

Add DiffusionGemma-26B-A4B export for Core AI on-device inference

Users can now export the Google DiffusionGemma-26B-A4B model to Core AI for on-device inference on macOS. The export produces a bundle containing an autoregressive encoder (which fills the KV cache) and a bidirectional decoder (which refines the token canvas via iterative denoising). The process supports 4-bit weight quantization and allows exporting the encoder-only or the full encoder-decoder bundle.

_models/diffusion\gemma · high confidence

Add Parakeet TDT live streaming transcription support

Introduces a new live transcription capability for the Parakeet TDT model architecture, enabling real-time, chunked audio processing with incremental text updates. This change adds the necessary Swift inference components, including a mel spectrogram front-end, a transducer decoder with streaming state management, and a streaming session API that handles windowed audio chunks, endpointing, and partial transcription delivery.

swift/Sources/CoreAISpeech · high confidence

Add Wan 2.1 text-to-video generation pipeline

Introduces a new Wan 2.1 text-to-video pipeline that orchestrates tokenization, text encoding, denoising, and 3D VAE decoding to generate video frames. The implementation includes a new \VideoPipeline\ protocol and \VideoConfiguration\ struct, supporting quality presets (fast, balanced, best) and tiled VAE decoding to manage memory usage for high-resolution outputs.

swift/Sources/CoreAIVideoDiffusionPipeline · high confidence

Add Wan 2.1 text-to-video generation tool

Introduces the \videodiffusion-runner\ command-line tool, enabling users to generate videos from text prompts using the Wan 2.1 diffusion model. The tool supports configurable parameters for video quality, resolution, frame count, and duration, and includes advanced features such as tiled VAE decoding to reduce memory usage, CFG cutoff optimization, and intermediate latent dumping for debugging.

swift/Sources/Tools/videodiffusion-runner · high confidence

Add latent-to-RGB preview and tuning utilities

Users can now generate fast, draft RGB previews from diffusion model latents without waiting for the full VAE decode. This is achieved via a new \LatentRGBCoefficients\ struct and an \NDArray.asRGB(coefficients:)\ method that applies a linear projection (using BLAS GEMM) to map latent channels to RGB. To ensure accuracy, a new \LatentPreviewTuner\ class allows users to collect latent/image pairs during generation and fit the specific projection coefficients for their model via least-squares regression, which can then be used for real-time previewing.

swift/Sources/CoreAIDiffusionPipeline/Preview · high confidence

Add video input and output support for vision-language models

Developers can now pass video files to vision-language models. The \VideoInput\ API extracts frames from local video files using configurable sampling strategies (uniform count or fixed frame rate) and delivers them lazily to keep memory usage low. Additionally, the \VideoWriter\ and \StreamingVideoWriter\ components allow encoding generated frames into MP4, GIF, APNG, or WebP formats, with streaming support for memory-efficient sequential writing.

swift/Sources/CoreAIShared/Video · high confidence

Introduce DiffusionBundle and FlowTransformerPipeline for FLUX.2 and Sana Sprint

The diffusion pipeline now uses a unified \DiffusionBundle\ to load model assets and configuration from \metadata.json\ (schema 0.2), replacing the previous descriptor-based loading. This change introduces \FlowTransformerPipeline\, which supports two new model families: FLUX.2 (using Qwen3 text encoding, in-graph RoPE, and flow-matching Euler sampling) and Sana Sprint (using Gemma2 text encoding, TrigFlow/SCM stochastic sampling, and instruction-prefixed prompts). The pipeline also adds latent preview capabilities via \LatentPreview\ and \LatentPreviewTuner\, allowing real-time RGB previews from latent tensors using fitted linear projection coefficients.

swift/Sources/CoreAIDiffusionPipeline/Pipelines · high confidence

Introduce OpenAI-compatible LLM inference server

The llm-server tool now provides an HTTP server that implements the OpenAI API interface, exposing endpoints for chat completions (/v1/chat/completions), text completions (/v1/completions), and model listing (/v1/models). The server supports both streaming and non-streaming generation, includes a /ready health check and a /v1/stats endpoint for monitoring performance metrics, and allows users to control inference behavior via command-line arguments for sampling parameters, KV cache strategies, and request queue depth. A --replay mode is also available to process JSONL request files directly for testing and benchmarking without starting the HTTP service. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; LLM server exposes memory estimates and batch occupancy on /ready, and supports auto concurrent requests; Unified stop-token resolution across text and VLM adapters; and 1 more.

swift/Sources/Tools/llm-server · medium confidence · unverified

Introduce SAM3 Video segmentation runtime

Adds the CoreAIVideoSegmenter module implementing the SAM3 Video model, which enables tracking and segmentation of multiple objects across video frames. The runtime includes a frame preprocessor for efficient tensor preparation, a memory bank packer to manage temporal context, and a tracking loop with an associator for matching detections to existing tracks. Post-processing handles mask cleanup and overlay rendering, while session management maintains object history and state across the video sequence.

swift/Sources/CoreAIVideoSegmenter · high confidence

Introduces core data models and concurrency controls for the LLM server

This change adds the foundational Swift types and infrastructure required for the new OpenAI-compatible LLM server. It defines request and response structures for both legacy completions and modern chat endpoints, including support for tool calls, reasoning effort controls, and reproducible generation seeds. Additionally, it introduces an async request queue to manage concurrency limits and a replay format to allow deterministic testing and debugging of server interactions. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; LLM server exposes memory estimates and batch occupancy on /ready, and supports auto concurrent requests.

swift/Sources/CoreAILMCommon · medium confidence · unverified

New model support and export infrastructure for Parakeet, Sana, Wan, Gemma 4, OLMo 2, and SAM3

This update adds export support for several new model families and introduces foundational export infrastructure. Users can now export the Parakeet live transcription model, the Sana Sprint 0.6B image generation model, and the Wan 2.1 text-to-video pipeline. iOS-specific exports are added for the Gemma 4 text decoder (supporting E2B/E4B variants with sliding-window attention) and the OLMo 2 model. Additionally, the SAM3 image segmentation model is re-authored for iOS with optimized primitives. The export system itself is enhanced with a new composite-op externalization module to preserve fused kernels (like RMSNorm and RoPE) and standardized constants for graph names and KV cache handling across iOS and macOS platforms. Also in this area: 1 commit adding/updating tests in python; 2 commits (0 fixes) modifying python; 1 commit (0 fixes) modifying python; SAM3 now masks prompt padding to match Hugging Face behavior; Fixes incorrect GPU gather results for large embedding tables in Gemma-3n-E4B; and 1 more.

python · medium confidence · unverified

New runtime utilities and expanded model structure support in CoreAIShared

The CoreAIShared/Runtime module introduces several new capabilities and structural changes. It adds a backpressured stream to cap unconsumed elements in async producers, a bilinear resampler for image resizing, a bitset for efficient mask operations, and a reader for NumPy .npy files. Model structure detection is expanded to support SAM3 multi-function and video segmenters, which now route to the NeuralEngine and GPU respectively. Asset resolution is unified into a filename-agnostic approach that discovers .aimodel/.aimodelc files, and a new CLI flag allows clearing the Core AI specialization cache. Additionally, NDArray helpers now handle non-contiguous layouts and specific scalar-type filling, while timing utilities and resource management protocols have been moved to this shared location.

swift/Sources/CoreAIShared/Runtime · high confidence

New sampling controls and reproducibility knobs

Users can now configure Min-P filtering, repetition penalty (with a configurable window), and a deterministic seed for reproducible generation. The sampling pipeline applies these in a defined order: repetition penalty, temperature, Min-P, Top-P, and Top-K. These options are available in both the CPU composite sampler and the GPU MPSGraph sampler, with the GPU engine supporting bitmask expansion for constrained generation. Also in this area: Speculative decoding support and metadata-driven diffusion asset loading.

swift/Sources/CoreAILanguageModels/Samplers · high confidence

New speech-recognizer tool with streaming, parity testing, and Parakeet v3 support

A new command-line tool, speech-recognizer, is introduced to transcribe audio using CoreAI speech models. It supports both offline and streaming transcription modes, including a --stream flag for incremental output and a --realtime flag to pace input for realistic latency measurement. The tool now supports Parakeet v3 models alongside Whisper, enabling features like encoder context truncation and silence-based predictor resets. A comprehensive --parity-test mode allows users to validate the Swift runtime's output against PyTorch reference traces for mel features, encoder hidden states, and tokens, with configurable PSNR and cosine similarity floors. The tool also includes diagnostic flags like --dump-mel for front-end verification and --clear-coreai-cache to force model re-specialization.

swift/Sources/Tools/speech-recognizer · high confidence

New video-segmenter CLI tool with parity-checking and tracking overrides

A new command-line tool (\video-segmenter\) is introduced to segment and track objects in videos using text prompts via the SAM 3 video model. Users can now run inference directly from the terminal, specifying input videos, text prompts, and output paths. The tool exposes granular control over tracking behavior through command-line flags, allowing overrides for detection thresholds, NMS settings, hole-filling areas, and hotstart delays. It also supports rendering annotated videos with configurable mask opacity, bounding boxes, and labels. For validation purposes, a \--parity\ mode is included, which replays a clip against a reference directory to report per-frame agreement on masks, scores, and boxes, ensuring the Swift implementation matches the expected behavior.

swift/Sources/Tools/video-segmenter · high confidence

SAM 3 Video Segmentation export and runtime

Added a new export script and documentation for SAM 3 Video Segmentation, enabling users to convert the model into a bundle format with support for 336, 672, and 1008 input resolutions. The entry point \models/sam3\_video/export.py\ wraps the shared segmentation export logic, specifically overriding dependencies to require \transformers\>=5.5.4\ and \huggingface-hub\>=1.5.0\ to satisfy the model's requirements, and includes a README detailing the pipeline functions, setup, and Swift runtime integration.

_models/sam3\video · high confidence

Support for Vision-Language Models and Per-Layer Embeddings

The language model bundle now supports Vision-Language Models (VLMs) by introducing a \VisionConfig\ field and requiring it for VLM bundles, enabling multimodal inference. Additionally, the system now supports externalized Per-Layer Embeddings (PLE) via a new \SafeTensorsReader\ and \PerLayerEmbeddings\ loader, allowing large INT8 embedding tables to be mapped from sidecar files rather than embedded in the graph. The bundle structure was also renamed from \LanguageBundle\ to \LanguageModelBundle\ to reflect this broader scope, and new configuration options for state handling, prefill chunking, and model-specific runtime overrides (such as sliding windows and RoPE parameters) were added to \LanguageConfig\.

swift/Sources/CoreAILanguageModels/Bundle · high confidence

Support for Vision-Language Models via FoundationModels protocol

Users can now load and run Vision-Language Models (VLMs) using the standard FoundationModels protocol. A new CoreAIVisionLanguageModel adapter allows loading VLM bundles (kind=vlm) and executing multi-modal prompts that include both text and image attachments. The implementation handles image encoding, tokenization, and streaming generation through the CoreAISequentialVLMEngine, enabling capabilities like image understanding within the existing language model session interface. Also in this area: 3 commits (0 fixes) modifying swift; Inference options split into orthogonal token and logits axes; Unified stop-token resolution across text and VLM adapters.

swift/Sources/CoreAILanguageModels/VLM · medium confidence · unverified

Behavioural changes

1 commit (0 fixes) modifying swift/Sources/CoreAIShared/RNG

A change to existing behaviour in swift/Sources/CoreAIShared/RNG — 1 commit, 5 files: “Consolidate random number generators into CoreAIShared (\#273) (\#421)” (refs \#273, \#421).

swift/Sources/CoreAIShared/RNG · medium confidence · unverified

1 commit (0 fixes) modifying swift/Sources/Tools/CoreAIMemoryMeasurement

A change to existing behaviour in swift/Sources/Tools/CoreAIMemoryMeasurement — 1 commit, 5 files: “Peak Memory Usage Measurement Tool (\#521)” (refs \#521).

swift/Sources/Tools/CoreAIMemoryMeasurement · medium confidence · unverified

12 commits (0 fixes) modifying swift/Sources/CoreAILanguageModels/SpeculativeDecoding

A change to existing behaviour in swift/Sources/CoreAILanguageModels/SpeculativeDecoding — 12 commits, 16 files: “Add the static-shape draft head and the engine hooks it reads (\#394)”; “Add a speculative decoding protocol, decoder and prompt-lookup drafter (\#378)”; “Speculative decoding: run the Gemma 4 draft head on the CPU (\#513)”; and 9 more (refs \#394, \#378, \#513, \#391, \#410, …).

swift/Sources/CoreAILanguageModels/SpeculativeDecoding · medium confidence · unverified

13 commits (0 fixes) modifying models/gemma4

A change to existing behaviour in models/gemma4 — 13 commits, 5 files: “Add note (\#326)”; “Move Gemma 4 PLE sidecar to auxiliary\_assets in metadata.json (\#332)”; “Add the static-shape draft head and the engine hooks it reads (\#394)”; and 10 more (refs \#326, \#332, \#394, \#513, \#301, …).

models/gemma4 · medium confidence · unverified

21 commits (4 fixes) modifying swift/Sources/CoreAILanguageModels/Handlers

A change to existing behaviour in swift/Sources/CoreAILanguageModels/Handlers — 21 commits (4 fixs), 18 files: “Support 8-bit KV-cache states and fix StateKind docs (\#356)”; “Optimize macOS LLMs prefill with distinct prompt function. (\#204)”; “Add bind(into:) for zero-copy state binding with loop-based insert (\#156)”; and 18 more (refs \#356, \#204, \#156, \#416, \#394, …).

swift/Sources/CoreAILanguageModels/Handlers · medium confidence · unverified

4 commits (0 fixes) modifying models/parakeet

A change to existing behaviour in models/parakeet — 4 commits, 2 files: “New Model: parakeet-tdt-0.6b-v2 export (\#401)”; “Live Transcription with Parakeet v3 (\#184)”; “Adding Parakeet Export and Runtime (\#136)”; and 1 more (refs \#401, \#184, \#136, \#193).

models/parakeet · medium confidence · unverified

7 commits (0 fixes) modifying models/clap, models/clip

A change to existing behaviour in models/clap, models/clip — 7 commits, 4 files: “Embedding modality: CLIP and CLAP bundles through coreai.embedding.export (\#395)”; “Default to release mode for exporting, with --include-debug-info opt-in (\#173)”; “Update Core AI Wheel Versions (\#291)”; and 4 more (refs \#395, \#173, \#291, \#358, \#336, …).

models/clap, models/clip · medium confidence · unverified

7 commits (1 fix) modifying models/efficient-sam

A change to existing behaviour in models/efficient-sam — 7 commits (1 fix), 2 files: “Default to release mode for exporting, with --include-debug-info opt-in (\#173)”; “Update Core AI Wheel Versions (\#291)”; “Remove prerelease allow (\#358)”; and 4 more (refs \#173, \#291, \#358, \#336, \#78, …).

models/efficient-sam · medium confidence · unverified

7 commits (2 fixes) modifying models/sam3

A change to existing behaviour in models/sam3 — 7 commits (2 fixs), 2 files: “Issue 381 Fix: Add attention mask to SAM3 export and runtime (\#385)”; “update sam3 CLI flags and default values with new CLI unit tests (\#167)”; “Remove prerelease allow (\#358)”; and 4 more (refs \#385, \#167, \#358, \#78, \#488, …).

models/sam3 · medium confidence · unverified

Batched inference and dynamic input sizing for object detection

The object detector now supports processing multiple images in a single batched forward pass via the new \detect(images:)\ API, improving throughput for multi-image scenarios. Additionally, \DetectionParameters\ now accepts \inputHeight\ and \inputWidth\ settings (defaulting to 800x800) to allow dynamic spatial dimensions for models that do not have fixed input shapes, and the \warmup\ method has been updated to accept these parameters to ensure the backend is prepared for the actual batch size and dimensions used in subsequent detection calls.

swift/Sources/CoreAIObjectDetector · high confidence

Benchmark tool adds prefill chunking, cache clearing, and timing metrics

The \llm-benchmark\ tool now supports configurable prefill chunking via \--chunk-size\ and \--chunk-threshold\ flags to optimize inference performance, and includes a new \--clear-coreai-cache\ flag to force re-specialization of Core AI models before loading. Additionally, the benchmark now reports detailed timing for model preparation and warmup phases, including cache hit status, in both console output and JSON reports, providing deeper insight into model load performance. Also in this area: 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; Core AI model asset validation now checks for valid model content, not just file existence.

swift/Sources/Tools/benchmark · medium confidence · unverified

Configurable image preprocessing strategies and enhanced tokenizer masks

Users can now select specific image preprocessing strategies (stretch, center-crop, or pad) when preparing images for vision-language models, with the system automatically handling layout transposition to the required CHW format. Additionally, the CLI text tokenizer now provides an attention mask alongside token IDs, allowing downstream components to distinguish between actual prompt content and padding tokens. Also in this area: 3 commits (0 fixes) modifying swift; Consolidate shared utilities and migrate model assets to auxiliary role.

swift/Sources/CoreAIShared/Image · medium confidence · unverified

Core AI diffusion components refactor: caching, input validation, and API expansion

The diffusion pipeline components have been updated to improve stability and performance. CoreAIDiffusionModelFunction now caches InferenceFunctions instead of reloading them per call, which prevents GPU memory leaks, and includes a fail-fast check for missing model assets. The public API has been expanded with a new ModelInput enum supporting cached NDArrays and a prepackInput helper for safer buffer handling, alongside strict validation to reject input buffers that do not match the model's expected dimensions. Additionally, CoreAIDenoiser and CoreAILatentCodec now use immutable views for tensor data, and documentation in TextEncoderOutput has been corrected to reflect T5 encoder behavior.

swift/Sources/CoreAIDiffusionPipeline/Components · high confidence

Core AI diffusion runner overhaul: new flags, model support, and latent tuning

The diffusion runner has been refactored to use the new DiffusionBundle API, dropping support for older Stable Diffusion models (1.5, 2.1, 3.5 Medium) and requiring the --model flag for all generation. New capabilities include a --clear-coreai-cache flag to force model re-specialization, a --reference-grid option for controlling token density in img2img, and a --guidance-mode flag to switch between distilled and manual classifier-free guidance. The tool now supports Flux.2 and Sana Sprint 0.6B models via the FlowTransformerPipeline. Additionally, a new latent preview tuning system allows users to collect per-step latents (--tune-preview) and fit RGB coefficients (--tune-fit) to optimize image generation, alongside a hidden --parity-test mode for validating component outputs against numpy fixtures.

swift/Sources/Tools/diffusion-runner · high confidence

Engine architecture refactored for idempotency, implicit prefix caching, and GPU-constrained generation

The inference engine implementation has been restructured to support independent, reusable generation sessions and more efficient processing. Session state (KV cache, token history) is now decoupled from the engine into a caller-owned \GenerationSessionState\, enabling the sequential engine to serve multiple independent conversations and allowing the pipelined engine to perform implicit prefix caching (automatically reusing cached prefixes across turns). A new \ConstrainedGenerationCapable\ protocol adds GPU-accelerated grammar-constrained sampling, and a \ChunkedPrefill\ loop optimizes prefill performance by splitting prompts into chunks, particularly when a dedicated prefill graph is available. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; LLM server exposes memory estimates and batch occupancy on /ready, and supports auto concurrent requests; Consolidate shared utilities and migrate model assets to auxiliary role; and 1 more.

swift/Sources/CoreAILanguageModels/InferenceEngines · medium confidence · unverified

Enhanced profiling granularity and refined performance metrics API

The profiling system now exposes five new fine-grained engine sub-spans—GatherEmbeddings, MaskBuild, RopeBuild, PLEGather, and LogitsCopy—allowing users to attribute per-token runner-side work more precisely in Instruments traces. Additionally, the PerformanceMetrics API has been refined: token counts are now exposed via a computed totalTokenCount property and recorded through recordPromptTokens and recordGeneratedTokens methods, replacing the previous setter-based approach for better encapsulation.

swift/Sources/CoreAILanguageModels/Profiling · high confidence

Expanded model support and new VLM export capability

The coreai-models package now supports exporting a wider range of models, including Qwen3-VL (vision-language), Muse Glimmer 30B (with optional speculative-decoding drafter), Phi-3/3.5/4-mini, SmolLM2, OLMo 2, Gemma 3n, and Gemma 4. A new VLM export script produces multi-asset bundles containing a text decoder, token-embedding lookup, and vision encoder. Diffusion support has shifted from Stable Diffusion (removed) to Flux 2 (including img2img), Sana Sprint, and Wan 2.1 text-to-video. The export pipeline now supports graph-mode quantization on macOS, configurable image preprocessing for VLMs, and a default release mode for smaller assets. iOS exports default to a 4096 max context length, and utility model platform filtering is now respected.

coreai-models · high confidence

Guided generation now supports rollback, jump-forward, and completion detection via XGrammar

The C bridge for the XGrammar library in swift/Sources/lib/CXGrammar has been extended to expose advanced guided-generation capabilities. Users can now roll back the grammar state by a specified number of tokens, retrieve the longest deterministic string for immediate output (jump-forward), and check if the grammar's root rule is fully completed. These features are implemented in the new xgrammar\_c\_bridge.cpp and updated xgrammar\_c\_bridge.h, replacing the previous header location and enabling more robust control over structured text generation.

swift/Sources · high confidence

Image segmenter CLI gains cache clearing and load timing

The image-segmenter tool now includes a --clear-coreai-cache flag that forces re-specialization of the Core AI model by clearing its cached specialization before loading. Additionally, the CLI now reports the model load time and indicates whether a cache hit occurred, helping users monitor performance. The implementation also adds macOS-specific guards for the file-opening helper and refactors bundle loading to use ImageSegmentationBundle instead of ModelBundle.

swift/Sources/Tools/image-segmenter · high confidence

Improved segmentation mask quality and centralized visualization logic

The segmentation postprocessor now uses bilinear upsampling instead of nearest-neighbor for both binary masks and probability grids, which eliminates staircase artifacts and produces smoother edges, particularly for low-resolution models like SAM3 lite. Additionally, visualization logic for heatmaps, instance colors, and HSV-to-RGB conversion has been moved to the shared \CoreAIShared\ module to reduce duplication.

swift/Sources/CoreAIImageSegmenter/Postprocessing · high confidence

Lazy model loading and agentic reasoning support

Model initialization now defaults to lazy loading, deferring the inference engine load until the first generation request, which reduces startup latency and memory usage; users can opt into eager loading via a new \LoadMode\ parameter. The system now supports agentic chain-of-thought formats, allowing models to emit reasoning content as structured messages rather than inline tags. Additionally, users can configure prefill chunk sizes and thresholds to optimize performance for specific hardware or model architectures.

swift/Sources/CoreAILanguageModels/LanguageModel · high confidence

Logger output redirected to stderr

The CLILogger now writes diagnostic messages to standard error instead of standard output. This ensures that stdout remains clean for programmatic output, such as JSONL results in replay mode, preventing log noise from interfering with structured data consumption.

swift/Sources/CoreAIShared/Logger · high confidence

Model exports default to release mode with optional debug info

The export scripts for CLAP, CLIP, Depth Anything, EDSR, Efficient SAM, PVT, RoBERTa, T5, and Wav2Vec2 now default to RELEASE mode when converting models to Core AI, producing smaller assets by embedding minimal debug information. Users can opt into DEBUG mode via the new --include-debug-info flag to embed full debug information for troubleshooting conversion issues. Additionally, the coreai-core and coreai-torch dependencies have been updated to versions 1.0.0b3 and 0.4.3 respectively, and the Efficient SAM export now defaults to 2 points per query to support box prompts.

(repo-wide) · high confidence

Object detector CLI now supports batch image processing and dynamic model inputs

The object detector command-line tool has been updated to accept multiple input images via the --image flag, enabling batch detection in a single run. When processing multiple images, the --output-image flag now expects a directory path where individual detection overlays are saved, and --output-json returns a structured array of results per image. Additionally, users can now override the model's input dimensions using --input-height and --input-width, which is required for dynamic Core AI models. A new --clear-coreai-cache flag allows forcing a re-specialization of the model by clearing the Core AI cache before loading.

swift/Sources/Tools/object-detector · high confidence

Refactored decoding strategies to use eager setup and async sequences

The decoding strategies in CoreAILanguageModels have been refactored to improve performance and API consistency. The \DecodingStrategy\ protocol now returns a typed \AsyncSequence\ (e.g., \VanillaDecodedSequence\, \ConstrainedDecodedSequence\) instead of a generic \AsyncThrowingStream\, with all session creation and tokenization performed eagerly before the sequence is returned. A new \PipelinedConstrainedDecodingStrategy\ has been added to enable GPU-accelerated grammar-constrained decoding by applying bitmasks directly in the MPSGraph sampler, avoiding CPU logit transfers. Additionally, \StopSequences\ now supports additional EOS token IDs from tokenizer configs, and log probability calculations utilize a new \LogProbabilities\ struct for better numerical stability.

swift/Sources/CoreAILanguageModels/DecodingStrategies · high confidence

Scheduler refactoring: removal of legacy schedulers and expansion of DiscreteFlowScheduler

The scheduler module has been significantly restructured. The PNDMScheduler and DPMSolverMultistepScheduler implementations have been removed, and the SchedulerType enum now only exposes discrete flow matching. The DiscreteFlowScheduler has been expanded to support Flux, Wan, and Sana Sprint models, introducing a new initializer for explicit sigma schedules (e.g., TrigFlow), a stochastic sampling step for consistency-distilled models, and a public currentSigma property to expose the denoised x0 estimate for live previews. Additionally, the PredictionType enum has been moved to the top-level Scheduler.swift file.

swift/Sources/CoreAIDiffusionPipeline/Schedulers · high confidence

Schema-constrained generation now limits whitespace and supports rollback/jump-forward

Guided JSON generation now prevents infinite whitespace loops by enforcing a default limit of 32 whitespace characters between tokens (configurable or disabled), which is applied via a new WhitespaceGuard that restricts the next-token mask. The ConstrainedGenerationSession also exposes rollback and jump-forward capabilities, allowing the generation engine to undo recent tokens and inject deterministic strings, while VocabularyType detection is improved to correctly identify byte-fallback tokenizers.

swift/Sources/CoreAILanguageModels/GuidedGeneration · high confidence

Streaming tool-call detection and grammar library cleanup

The CoreAILanguageModels module now includes a streaming parser (ToolCallParser) that detects tool-call blocks in the model's token stream, supporting JSON, ATEM, and Qwen3-Coder XML formats. A helper function (lastSafeIndex) ensures streaming markers are not split across deltas. Additionally, unused model shape configuration files and the bundled CXGrammar (XGrammar) framework headers have been removed from this location.

swift/Sources/CoreAILanguageModels · high confidence

Support for multi-function SAM3 Lite model assets

The image segmentation engine now automatically detects and supports two distinct model asset structures: the existing single-function models (like baseline SAM3 and EfficientSAM) and new multi-function models (produced by the SAM3 lite export). For multi-function assets, the engine dynamically wires together three separate graphs—image\_encode, text\_encode, and detect—at runtime to produce the same SegmentationOutput shape as before. This change allows users to leverage optimized SAM3 Lite models without code changes, while point-query support remains exclusive to single-function models.

swift/Sources/CoreAIImageSegmenter · high confidence

Support for new model kinds and improved bundle validation

The model bundle system now recognizes \videoDiffusion\, \videoSegmenter\, and \speechRecognizer\ kinds, enabling the runtime to handle video diffusion (Wan), SAM 3 video segmentation, and speech recognition models. The \LanguageBundle\ type has been renamed to \LanguageModelBundle\ to reflect its scope. Additionally, bundle loading now validates that the input is a directory rather than a direct model asset file (\.aimodel\/\.aimodelc\), providing a clear error message if a user points the runner at a single asset file. The asset verification method was also renamed to \verifyAssetsExisting\ for clarity, and error messages were updated to provide more specific guidance on missing assets and bundle structure.

swift/Sources/CoreAIShared/Bundle · high confidence

Vectorized log-probability computation and enhanced generation API

Text generation now includes a new LogProbabilities struct that uses the Accelerate framework for vectorized log-softmax calculations, significantly improving performance for per-token log probability and top-K alternative extraction. The TextGenerator API has been updated to return token IDs alongside text and logits in generateWithLogits, and a new evaluateRawTokens method allows direct evaluation of pre-tokenized token sequences. Additionally, the decoding strategy is now accessed asynchronously via try await to support serialization of back-to-back turns.

swift/Sources/CoreAILanguageModels/TextGeneration · high confidence

Whisper export defaults to release mode with optional debug info and dynamic shapes

The Whisper model export now defaults to release mode, producing smaller assets by embedding minimal debug information; users can opt into full debug info via the new --include-debug-info flag. Additionally, the export process now traces the model with dynamic shapes (specifically for decoder input sequence length), which improves flexibility for variable-length inputs during inference.

models/whisper · high confidence

Yolo export defaults to release mode with optional debug info and updated Core AI dependencies

The Yolo model export process now defaults to release mode, resulting in smaller exported assets, while adding a --include-debug-info flag to opt-in to debug information embedding. This change also updates the coreai-core and coreai-torch dependencies to versions 1.0.0b3 and 0.4.3 respectively. Additionally, the documentation reflects new capabilities for batched detection and dynamic shape handling in both iOS/macOS applications and the Mac command-line tool, including support for explicit input dimensions and kernel warmup.

models/yolo · high confidence

llm-runner adds video input, MinP sampling, repetition penalties, and VLM image options

The llm-runner tool now supports vision-language models with new command-line flags for video input (--video, --video-frames, --video-sampling) and configurable image preprocessing (--image, --image-strategy, --image-info). It introduces MinP sampling (--min-p) alongside existing Top-P, and adds repetition penalty controls (--repetition-penalty, --repetition-penalty-window) with validation against constrained generation. Prefill chunking is refined with a separate --chunk-threshold option and a default of 4096 tokens on macOS with \>=64GB memory. A new --clear-coreai-cache flag forces re-specialization of the Core AI model cache, and the internal bundle type has been renamed from LanguageBundle to LanguageModelBundle. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; Unified stop-token resolution across text and VLM adapters; Core AI model asset validation now checks for valid model content, not just file existence.

swift/Sources/Tools/llm-runner · medium confidence · unverified

Fixes

Fixes token count mismatch error in logits output handling

The \LogitsWriter\ now accepts generated token IDs directly instead of decoding generated text to derive them, which resolves a token count mismatch error that occurred when the encoded token count differed from the logits count. This change ensures that the validation logic correctly compares the number of provided token IDs against the number of logits vectors, preventing crashes or invalid states during logits saving or console printing.

swift/Sources/CoreAILanguageModels/Output · high confidence

Test coverage

5 commits adding/updating tests in swift/Tests/GuidedGenerationTests; Added comprehensive test coverage for CoreAISpeech audio processing and configuration; Added test coverage for CoreAIShared utilities; Added test coverage for CoreAIVideoSegmenter components; Added test coverage for diffusion pipeline configuration, token math, and input validation; Added tests for ImageSegmentationBundle and updated segmentation engine tests; Added tests for ObjectDetector batch planning logic; Added tests for language model parsing, engine lifecycle, and configuration; Added tests for video input, reading, and writing in CoreAIShared; Added unit tests for CoreAILMCommon API types and server utilities; Removed unused XCTest plan file.

Dependencies

New video, speech, and LLM server capabilities with dependency updates

This release introduces new Swift libraries and tools for video diffusion (CoreAIVideoDiffusionPipeline), video segmentation (CoreAIVideoSegmenter), and speech recognition (CoreAISpeech), alongside a new llm-server executable for OpenAI-compatible LLM serving. The CXGrammar dependency is now a source-based target using XGrammar v0.2.2 instead of a binary framework. Python dependencies are updated, including transformers to v5.12.1, huggingface-hub to \>=1.5.0, and tokenizers capped at \<0.23, with new CLI scripts for VLM export and a flaky test marker added.

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 60 → 64 (+3.7)
  • Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.

Lenses

  • Code Health 66 → 76 (+10.4)
  • Architecture 92 → 93 (+1.4)
  • Maturity 58 → 58 (+0.2)
  • Readiness 54 → 56 (+1.9)
  • Security 75 → 83 (+8.7)
  • Performance 83 → 100 (+17.1)

Resolved (62)

  • Change-coupling hub: LLMRunnerMain.swift → CoreAISequentialVLMEngine.swift, CoreAIVisionLanguageModel.swift, LLMServerMain.swift (swift/Sources/Tools/llm-runner/LLMRunnerMain.swift)
  • ClassTooLong: CoreAISequentialVLMEngine (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialVLMEngine.swift)
  • CoreAISequentialVLMEngine.init (cognitive 27) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialVLMEngine.swift)
  • CoreAISequentialVLMEngine.init (cyclomatic 21) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialVLMEngine.swift)
  • Duplicated block (10 lines × 10) (models/clap/export.py)
  • Duplicated block (10 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/RNG/NumPyRandomSource.swift)
  • Duplicated block (11 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/Components/CoreAILatentCodec.swift)
  • Duplicated block (11 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/RNG/NumPyRandomSource.swift)
  • Duplicated block (12 lines × 2) (python/src/coreai_models/export/pipeline.py)
  • Duplicated block (12 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/RNG/NumPyRandomSource.swift)
  • Duplicated block (12 lines × 2) (swift/Sources/CoreAILanguageModels/Bundle/LanguageConfig.swift)
  • Duplicated block (12 lines × 2) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAIPipelinedEngine.swift)
  • Duplicated block (12 lines × 6) (swift/Sources/CoreAIDiffusionPipeline/Bundle/DiffusionConfig.swift)
  • Duplicated block (12 lines × 7) (swift/Sources/CoreAIDiffusionPipeline/Bundle/DiffusionConfig.swift)
  • Duplicated block (15–16 lines × 2) (swift/Sources/CoreAILanguageModels/TextGeneration/TextGenerator.swift)
  • Duplicated block (17–18 lines × 2) (python/src/coreai_models/models/macos/muse_glimmer_drafter_dflash.py)
  • Duplicated block (19 lines × 4) (models/clap/export.py)
  • Duplicated block (19–20 lines × 3) (models/clap/export.py)
  • Duplicated block (21–23 lines × 10) (models/clap/export.py)
  • Duplicated block (22–23 lines × 7) (models/clip/export.py)
  • …and 42 more

New (218)

  • BaseForCausalLM.from_hf_memory_efficient (cognitive 16) (python/src/coreai_models/models/base.py)
  • BatchScheduler.pump (cyclomatic 21) (swift/Sources/CoreAILanguageModels/InferenceEngines/BatchScheduler.swift)
  • CPUDrafter.readSharedKV (cognitive 31) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/CPUDrafter.swift)
  • CPUDrafter.readSharedKV (cyclomatic 18) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/CPUDrafter.swift)
  • Change coupling: CoreAIVisionLanguageModel.swift ↔ LLMRunnerMain.swift (swift/Sources/CoreAILanguageModels/VLM/CoreAIVisionLanguageModel.swift)
  • ChatHandler.swift.tokenizeMessages::render (cognitive 30) (swift/Sources/Tools/llm-server/ChatHandler.swift)
  • ChatHandler.swift.tokenizeMessages::render (cyclomatic 16) (swift/Sources/Tools/llm-server/ChatHandler.swift)
  • CoreAISequentialEngine.generationResetPlan (cognitive 16) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialEngine.swift)
  • DiffusionRunner.resolveOutputSize (cognitive 17) (swift/Sources/Tools/diffusion-runner/DiffusionRunnerMain.swift)
  • DraftPack.require (cognitive 19) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
  • DraftPack.require (cyclomatic 17) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
  • DraftPack.validate (cognitive 29) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
  • DraftPack.validate (cyclomatic 20) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
  • Duplicated block (10 lines × 2) (swift/Sources/CoreAIShared/RNG/NumPyRandomSource.swift)
  • Duplicated block (11 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/Components/CoreAILatentCodec.swift)
  • Duplicated block (12 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/Pipelines/FlowTransformerPipeline+Resources.swift)
  • Duplicated block (12 lines × 2) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAIPipelinedEngine.swift)
  • Duplicated block (15–16 lines × 2) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/CPUDrafter.swift)
  • Duplicated block (19 lines × 3) (models/parakeet/export.py)
  • Duplicated block (21–23 lines × 8) (models/depth-anything/export.py)
  • …and 198 more

Changes since last survey

  • 60 commits — 55 feature/other, 5 fixes

By area

  • swift/Sources — 31 commits
  • python/src — 12 commits
  • python/tests — 3 commits
  • (root) — 2 commits
  • .github/workflows — 2 commits
  • models/README.md — 2 commits
  • models/gemma4 — 2 commits
  • skills/skills — 2 commits
  • .github/scripts — 1 commit
  • models/efficient-sam — 1 commit
  • models/parakeet — 1 commit
  • swift/Tests — 1 commit

Notable commits

  • fix: Fix graph mode quantization for gemma3n models (#407)
  • fix: Fix graph mode quantization for qwen3_5 models (#389)
  • fix: Fix slow logits top-k extraction in LogitsWriter (#512)
  • fix: Fix the segmenter module name in the README samples (#488)
  • fix: Fix two stale snippets in the skills docs (#431)
  • change: Run several prompts at once in llm-runner and llm-benchmark, and refuse batched requests past the device's memory recommendation (#511)
  • change: quantize on a chosen device and externalize composites for every backend (#408)
  • change: Add EmbeddingGemma 2 export: coreai.embedding.export and registry entry (#369)
  • change: Add native quantized weight modules for Gemma 4 (#419)
  • change: Add the static-shape draft head and the engine hooks it reads (#394)
  • change: Consolidate random number generators into CoreAIShared (#273) (#421)
  • change: Constrained session tests go through ConstrainedSessionSlot (#526)
  • change: DiffusionGemma: keep RoPE inv_freq buffer in float32 under to(dtype) (#482) (#483)
  • change: Document speculative decoding in llm-runner (#403)
  • change: Document the real engine variants (#489)
  • change: Draft head: start reading the scan table in at load (#410)
  • change: Dynamic Flux2 (#510)
  • change: Embedding modality: CLIP and CLAP bundles through coreai.embedding.export (#395)
  • change: Fold the VLM engine into CoreAISequentialEngine (#388)
  • change: Follow-ups to the VLM fold (#416)
  • …and 40 more

Architecture

  • Unchanged — 0 containers · 2 contexts · 0 edges

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

apple/coreai-models was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 15 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 128d2f30c935d27807d0e41a0f22a985f5d38250 — the exact code this score is about.
  • Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-3c33c1882dc1.