apple/coreai-models
64.2
Adequate · 15 November 2026
85.5k
lines of production code
Swift
with Python
5
measurements over time
What this system is
Features
Add DiffusionGemma-26B-A4B export for Core AI on-device inference
Users can now export the Google DiffusionGemma-26B-A4B model to Core AI for on-device inference on macOS. The export produces a bundle containing an autoregressive encoder (which fills the KV cache) and a bidirectional decoder (which refines the token canvas via iterative denoising). The process supports 4-bit weight quantization and allows exporting the encoder-only or the full encoder-decoder bundle.
_models/diffusion\gemma · high confidence
Add Parakeet TDT live streaming transcription support
Introduces a new live transcription capability for the Parakeet TDT model architecture, enabling real-time, chunked audio processing with incremental text updates. This change adds the necessary Swift inference components, including a mel spectrogram front-end, a transducer decoder with streaming state management, and a streaming session API that handles windowed audio chunks, endpointing, and partial transcription delivery.
swift/Sources/CoreAISpeech · high confidence
Add Wan 2.1 text-to-video generation pipeline
Introduces a new Wan 2.1 text-to-video pipeline that orchestrates tokenization, text encoding, denoising, and 3D VAE decoding to generate video frames. The implementation includes a new \VideoPipeline\ protocol and \VideoConfiguration\ struct, supporting quality presets (fast, balanced, best) and tiled VAE decoding to manage memory usage for high-resolution outputs.
swift/Sources/CoreAIVideoDiffusionPipeline · high confidence
Add Wan 2.1 text-to-video generation tool
Introduces the \videodiffusion-runner\ command-line tool, enabling users to generate videos from text prompts using the Wan 2.1 diffusion model. The tool supports configurable parameters for video quality, resolution, frame count, and duration, and includes advanced features such as tiled VAE decoding to reduce memory usage, CFG cutoff optimization, and intermediate latent dumping for debugging.
swift/Sources/Tools/videodiffusion-runner · high confidence
Add latent-to-RGB preview and tuning utilities
Users can now generate fast, draft RGB previews from diffusion model latents without waiting for the full VAE decode. This is achieved via a new \LatentRGBCoefficients\ struct and an \NDArray.asRGB(coefficients:)\ method that applies a linear projection (using BLAS GEMM) to map latent channels to RGB. To ensure accuracy, a new \LatentPreviewTuner\ class allows users to collect latent/image pairs during generation and fit the specific projection coefficients for their model via least-squares regression, which can then be used for real-time previewing.
swift/Sources/CoreAIDiffusionPipeline/Preview · high confidence
Add video input and output support for vision-language models
Developers can now pass video files to vision-language models. The \VideoInput\ API extracts frames from local video files using configurable sampling strategies (uniform count or fixed frame rate) and delivers them lazily to keep memory usage low. Additionally, the \VideoWriter\ and \StreamingVideoWriter\ components allow encoding generated frames into MP4, GIF, APNG, or WebP formats, with streaming support for memory-efficient sequential writing.
swift/Sources/CoreAIShared/Video · high confidence
Introduce DiffusionBundle and FlowTransformerPipeline for FLUX.2 and Sana Sprint
The diffusion pipeline now uses a unified \DiffusionBundle\ to load model assets and configuration from \metadata.json\ (schema 0.2), replacing the previous descriptor-based loading. This change introduces \FlowTransformerPipeline\, which supports two new model families: FLUX.2 (using Qwen3 text encoding, in-graph RoPE, and flow-matching Euler sampling) and Sana Sprint (using Gemma2 text encoding, TrigFlow/SCM stochastic sampling, and instruction-prefixed prompts). The pipeline also adds latent preview capabilities via \LatentPreview\ and \LatentPreviewTuner\, allowing real-time RGB previews from latent tensors using fitted linear projection coefficients.
swift/Sources/CoreAIDiffusionPipeline/Pipelines · high confidence
Introduce OpenAI-compatible LLM inference server
The llm-server tool now provides an HTTP server that implements the OpenAI API interface, exposing endpoints for chat completions (/v1/chat/completions), text completions (/v1/completions), and model listing (/v1/models). The server supports both streaming and non-streaming generation, includes a /ready health check and a /v1/stats endpoint for monitoring performance metrics, and allows users to control inference behavior via command-line arguments for sampling parameters, KV cache strategies, and request queue depth. A --replay mode is also available to process JSONL request files directly for testing and benchmarking without starting the HTTP service. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; LLM server exposes memory estimates and batch occupancy on /ready, and supports auto concurrent requests; Unified stop-token resolution across text and VLM adapters; and 1 more.
swift/Sources/Tools/llm-server · medium confidence · unverified
Introduce SAM3 Video segmentation runtime
Adds the CoreAIVideoSegmenter module implementing the SAM3 Video model, which enables tracking and segmentation of multiple objects across video frames. The runtime includes a frame preprocessor for efficient tensor preparation, a memory bank packer to manage temporal context, and a tracking loop with an associator for matching detections to existing tracks. Post-processing handles mask cleanup and overlay rendering, while session management maintains object history and state across the video sequence.
swift/Sources/CoreAIVideoSegmenter · high confidence
Introduces core data models and concurrency controls for the LLM server
This change adds the foundational Swift types and infrastructure required for the new OpenAI-compatible LLM server. It defines request and response structures for both legacy completions and modern chat endpoints, including support for tool calls, reasoning effort controls, and reproducible generation seeds. Additionally, it introduces an async request queue to manage concurrency limits and a replay format to allow deterministic testing and debugging of server interactions. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; LLM server exposes memory estimates and batch occupancy on /ready, and supports auto concurrent requests.
swift/Sources/CoreAILMCommon · medium confidence · unverified
New model support and export infrastructure for Parakeet, Sana, Wan, Gemma 4, OLMo 2, and SAM3
This update adds export support for several new model families and introduces foundational export infrastructure. Users can now export the Parakeet live transcription model, the Sana Sprint 0.6B image generation model, and the Wan 2.1 text-to-video pipeline. iOS-specific exports are added for the Gemma 4 text decoder (supporting E2B/E4B variants with sliding-window attention) and the OLMo 2 model. Additionally, the SAM3 image segmentation model is re-authored for iOS with optimized primitives. The export system itself is enhanced with a new composite-op externalization module to preserve fused kernels (like RMSNorm and RoPE) and standardized constants for graph names and KV cache handling across iOS and macOS platforms. Also in this area: 1 commit adding/updating tests in python; 2 commits (0 fixes) modifying python; 1 commit (0 fixes) modifying python; SAM3 now masks prompt padding to match Hugging Face behavior; Fixes incorrect GPU gather results for large embedding tables in Gemma-3n-E4B; and 1 more.
python · medium confidence · unverified
New runtime utilities and expanded model structure support in CoreAIShared
The CoreAIShared/Runtime module introduces several new capabilities and structural changes. It adds a backpressured stream to cap unconsumed elements in async producers, a bilinear resampler for image resizing, a bitset for efficient mask operations, and a reader for NumPy .npy files. Model structure detection is expanded to support SAM3 multi-function and video segmenters, which now route to the NeuralEngine and GPU respectively. Asset resolution is unified into a filename-agnostic approach that discovers .aimodel/.aimodelc files, and a new CLI flag allows clearing the Core AI specialization cache. Additionally, NDArray helpers now handle non-contiguous layouts and specific scalar-type filling, while timing utilities and resource management protocols have been moved to this shared location.
swift/Sources/CoreAIShared/Runtime · high confidence
New sampling controls and reproducibility knobs
Users can now configure Min-P filtering, repetition penalty (with a configurable window), and a deterministic seed for reproducible generation. The sampling pipeline applies these in a defined order: repetition penalty, temperature, Min-P, Top-P, and Top-K. These options are available in both the CPU composite sampler and the GPU MPSGraph sampler, with the GPU engine supporting bitmask expansion for constrained generation. Also in this area: Speculative decoding support and metadata-driven diffusion asset loading.
swift/Sources/CoreAILanguageModels/Samplers · high confidence
New speech-recognizer tool with streaming, parity testing, and Parakeet v3 support
A new command-line tool, speech-recognizer, is introduced to transcribe audio using CoreAI speech models. It supports both offline and streaming transcription modes, including a --stream flag for incremental output and a --realtime flag to pace input for realistic latency measurement. The tool now supports Parakeet v3 models alongside Whisper, enabling features like encoder context truncation and silence-based predictor resets. A comprehensive --parity-test mode allows users to validate the Swift runtime's output against PyTorch reference traces for mel features, encoder hidden states, and tokens, with configurable PSNR and cosine similarity floors. The tool also includes diagnostic flags like --dump-mel for front-end verification and --clear-coreai-cache to force model re-specialization.
swift/Sources/Tools/speech-recognizer · high confidence
New video-segmenter CLI tool with parity-checking and tracking overrides
A new command-line tool (\video-segmenter\) is introduced to segment and track objects in videos using text prompts via the SAM 3 video model. Users can now run inference directly from the terminal, specifying input videos, text prompts, and output paths. The tool exposes granular control over tracking behavior through command-line flags, allowing overrides for detection thresholds, NMS settings, hole-filling areas, and hotstart delays. It also supports rendering annotated videos with configurable mask opacity, bounding boxes, and labels. For validation purposes, a \--parity\ mode is included, which replays a clip against a reference directory to report per-frame agreement on masks, scores, and boxes, ensuring the Swift implementation matches the expected behavior.
swift/Sources/Tools/video-segmenter · high confidence
SAM 3 Video Segmentation export and runtime
Added a new export script and documentation for SAM 3 Video Segmentation, enabling users to convert the model into a bundle format with support for 336, 672, and 1008 input resolutions. The entry point \models/sam3\_video/export.py\ wraps the shared segmentation export logic, specifically overriding dependencies to require \transformers\>=5.5.4\ and \huggingface-hub\>=1.5.0\ to satisfy the model's requirements, and includes a README detailing the pipeline functions, setup, and Swift runtime integration.
_models/sam3\video · high confidence
Support for Vision-Language Models and Per-Layer Embeddings
The language model bundle now supports Vision-Language Models (VLMs) by introducing a \VisionConfig\ field and requiring it for VLM bundles, enabling multimodal inference. Additionally, the system now supports externalized Per-Layer Embeddings (PLE) via a new \SafeTensorsReader\ and \PerLayerEmbeddings\ loader, allowing large INT8 embedding tables to be mapped from sidecar files rather than embedded in the graph. The bundle structure was also renamed from \LanguageBundle\ to \LanguageModelBundle\ to reflect this broader scope, and new configuration options for state handling, prefill chunking, and model-specific runtime overrides (such as sliding windows and RoPE parameters) were added to \LanguageConfig\.
swift/Sources/CoreAILanguageModels/Bundle · high confidence
Support for Vision-Language Models via FoundationModels protocol
Users can now load and run Vision-Language Models (VLMs) using the standard FoundationModels protocol. A new CoreAIVisionLanguageModel adapter allows loading VLM bundles (kind=vlm) and executing multi-modal prompts that include both text and image attachments. The implementation handles image encoding, tokenization, and streaming generation through the CoreAISequentialVLMEngine, enabling capabilities like image understanding within the existing language model session interface. Also in this area: 3 commits (0 fixes) modifying swift; Inference options split into orthogonal token and logits axes; Unified stop-token resolution across text and VLM adapters.
swift/Sources/CoreAILanguageModels/VLM · medium confidence · unverified
Behavioural changes
1 commit (0 fixes) modifying swift/Sources/CoreAIShared/RNG
A change to existing behaviour in swift/Sources/CoreAIShared/RNG — 1 commit, 5 files: “Consolidate random number generators into CoreAIShared (\#273) (\#421)” (refs \#273, \#421).
swift/Sources/CoreAIShared/RNG · medium confidence · unverified
1 commit (0 fixes) modifying swift/Sources/Tools/CoreAIMemoryMeasurement
A change to existing behaviour in swift/Sources/Tools/CoreAIMemoryMeasurement — 1 commit, 5 files: “Peak Memory Usage Measurement Tool (\#521)” (refs \#521).
swift/Sources/Tools/CoreAIMemoryMeasurement · medium confidence · unverified
12 commits (0 fixes) modifying swift/Sources/CoreAILanguageModels/SpeculativeDecoding
A change to existing behaviour in swift/Sources/CoreAILanguageModels/SpeculativeDecoding — 12 commits, 16 files: “Add the static-shape draft head and the engine hooks it reads (\#394)”; “Add a speculative decoding protocol, decoder and prompt-lookup drafter (\#378)”; “Speculative decoding: run the Gemma 4 draft head on the CPU (\#513)”; and 9 more (refs \#394, \#378, \#513, \#391, \#410, …).
swift/Sources/CoreAILanguageModels/SpeculativeDecoding · medium confidence · unverified
13 commits (0 fixes) modifying models/gemma4
A change to existing behaviour in models/gemma4 — 13 commits, 5 files: “Add note (\#326)”; “Move Gemma 4 PLE sidecar to auxiliary\_assets in metadata.json (\#332)”; “Add the static-shape draft head and the engine hooks it reads (\#394)”; and 10 more (refs \#326, \#332, \#394, \#513, \#301, …).
models/gemma4 · medium confidence · unverified
21 commits (4 fixes) modifying swift/Sources/CoreAILanguageModels/Handlers
A change to existing behaviour in swift/Sources/CoreAILanguageModels/Handlers — 21 commits (4 fixs), 18 files: “Support 8-bit KV-cache states and fix StateKind docs (\#356)”; “Optimize macOS LLMs prefill with distinct prompt function. (\#204)”; “Add bind(into:) for zero-copy state binding with loop-based insert (\#156)”; and 18 more (refs \#356, \#204, \#156, \#416, \#394, …).
swift/Sources/CoreAILanguageModels/Handlers · medium confidence · unverified
4 commits (0 fixes) modifying models/parakeet
A change to existing behaviour in models/parakeet — 4 commits, 2 files: “New Model: parakeet-tdt-0.6b-v2 export (\#401)”; “Live Transcription with Parakeet v3 (\#184)”; “Adding Parakeet Export and Runtime (\#136)”; and 1 more (refs \#401, \#184, \#136, \#193).
models/parakeet · medium confidence · unverified
7 commits (0 fixes) modifying models/clap, models/clip
A change to existing behaviour in models/clap, models/clip — 7 commits, 4 files: “Embedding modality: CLIP and CLAP bundles through coreai.embedding.export (\#395)”; “Default to release mode for exporting, with --include-debug-info opt-in (\#173)”; “Update Core AI Wheel Versions (\#291)”; and 4 more (refs \#395, \#173, \#291, \#358, \#336, …).
models/clap, models/clip · medium confidence · unverified
7 commits (1 fix) modifying models/efficient-sam
A change to existing behaviour in models/efficient-sam — 7 commits (1 fix), 2 files: “Default to release mode for exporting, with --include-debug-info opt-in (\#173)”; “Update Core AI Wheel Versions (\#291)”; “Remove prerelease allow (\#358)”; and 4 more (refs \#173, \#291, \#358, \#336, \#78, …).
models/efficient-sam · medium confidence · unverified
7 commits (2 fixes) modifying models/sam3
A change to existing behaviour in models/sam3 — 7 commits (2 fixs), 2 files: “Issue 381 Fix: Add attention mask to SAM3 export and runtime (\#385)”; “update sam3 CLI flags and default values with new CLI unit tests (\#167)”; “Remove prerelease allow (\#358)”; and 4 more (refs \#385, \#167, \#358, \#78, \#488, …).
models/sam3 · medium confidence · unverified
Batched inference and dynamic input sizing for object detection
The object detector now supports processing multiple images in a single batched forward pass via the new \detect(images:)\ API, improving throughput for multi-image scenarios. Additionally, \DetectionParameters\ now accepts \inputHeight\ and \inputWidth\ settings (defaulting to 800x800) to allow dynamic spatial dimensions for models that do not have fixed input shapes, and the \warmup\ method has been updated to accept these parameters to ensure the backend is prepared for the actual batch size and dimensions used in subsequent detection calls.
swift/Sources/CoreAIObjectDetector · high confidence
Benchmark tool adds prefill chunking, cache clearing, and timing metrics
The \llm-benchmark\ tool now supports configurable prefill chunking via \--chunk-size\ and \--chunk-threshold\ flags to optimize inference performance, and includes a new \--clear-coreai-cache\ flag to force re-specialization of Core AI models before loading. Additionally, the benchmark now reports detailed timing for model preparation and warmup phases, including cache hit status, in both console output and JSON reports, providing deeper insight into model load performance. Also in this area: 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; Core AI model asset validation now checks for valid model content, not just file existence.
swift/Sources/Tools/benchmark · medium confidence · unverified
Configurable image preprocessing strategies and enhanced tokenizer masks
Users can now select specific image preprocessing strategies (stretch, center-crop, or pad) when preparing images for vision-language models, with the system automatically handling layout transposition to the required CHW format. Additionally, the CLI text tokenizer now provides an attention mask alongside token IDs, allowing downstream components to distinguish between actual prompt content and padding tokens. Also in this area: 3 commits (0 fixes) modifying swift; Consolidate shared utilities and migrate model assets to auxiliary role.
swift/Sources/CoreAIShared/Image · medium confidence · unverified
Core AI diffusion components refactor: caching, input validation, and API expansion
The diffusion pipeline components have been updated to improve stability and performance. CoreAIDiffusionModelFunction now caches InferenceFunctions instead of reloading them per call, which prevents GPU memory leaks, and includes a fail-fast check for missing model assets. The public API has been expanded with a new ModelInput enum supporting cached NDArrays and a prepackInput helper for safer buffer handling, alongside strict validation to reject input buffers that do not match the model's expected dimensions. Additionally, CoreAIDenoiser and CoreAILatentCodec now use immutable views for tensor data, and documentation in TextEncoderOutput has been corrected to reflect T5 encoder behavior.
swift/Sources/CoreAIDiffusionPipeline/Components · high confidence
Core AI diffusion runner overhaul: new flags, model support, and latent tuning
The diffusion runner has been refactored to use the new DiffusionBundle API, dropping support for older Stable Diffusion models (1.5, 2.1, 3.5 Medium) and requiring the --model flag for all generation. New capabilities include a --clear-coreai-cache flag to force model re-specialization, a --reference-grid option for controlling token density in img2img, and a --guidance-mode flag to switch between distilled and manual classifier-free guidance. The tool now supports Flux.2 and Sana Sprint 0.6B models via the FlowTransformerPipeline. Additionally, a new latent preview tuning system allows users to collect per-step latents (--tune-preview) and fit RGB coefficients (--tune-fit) to optimize image generation, alongside a hidden --parity-test mode for validating component outputs against numpy fixtures.
swift/Sources/Tools/diffusion-runner · high confidence
Engine architecture refactored for idempotency, implicit prefix caching, and GPU-constrained generation
The inference engine implementation has been restructured to support independent, reusable generation sessions and more efficient processing. Session state (KV cache, token history) is now decoupled from the engine into a caller-owned \GenerationSessionState\, enabling the sequential engine to serve multiple independent conversations and allowing the pipelined engine to perform implicit prefix caching (automatically reusing cached prefixes across turns). A new \ConstrainedGenerationCapable\ protocol adds GPU-accelerated grammar-constrained sampling, and a \ChunkedPrefill\ loop optimizes prefill performance by splitting prompts into chunks, particularly when a dedicated prefill graph is available. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; LLM server exposes memory estimates and batch occupancy on /ready, and supports auto concurrent requests; Consolidate shared utilities and migrate model assets to auxiliary role; and 1 more.
swift/Sources/CoreAILanguageModels/InferenceEngines · medium confidence · unverified
Enhanced profiling granularity and refined performance metrics API
The profiling system now exposes five new fine-grained engine sub-spans—GatherEmbeddings, MaskBuild, RopeBuild, PLEGather, and LogitsCopy—allowing users to attribute per-token runner-side work more precisely in Instruments traces. Additionally, the PerformanceMetrics API has been refined: token counts are now exposed via a computed totalTokenCount property and recorded through recordPromptTokens and recordGeneratedTokens methods, replacing the previous setter-based approach for better encapsulation.
swift/Sources/CoreAILanguageModels/Profiling · high confidence
Expanded model support and new VLM export capability
The coreai-models package now supports exporting a wider range of models, including Qwen3-VL (vision-language), Muse Glimmer 30B (with optional speculative-decoding drafter), Phi-3/3.5/4-mini, SmolLM2, OLMo 2, Gemma 3n, and Gemma 4. A new VLM export script produces multi-asset bundles containing a text decoder, token-embedding lookup, and vision encoder. Diffusion support has shifted from Stable Diffusion (removed) to Flux 2 (including img2img), Sana Sprint, and Wan 2.1 text-to-video. The export pipeline now supports graph-mode quantization on macOS, configurable image preprocessing for VLMs, and a default release mode for smaller assets. iOS exports default to a 4096 max context length, and utility model platform filtering is now respected.
coreai-models · high confidence
Guided generation now supports rollback, jump-forward, and completion detection via XGrammar
The C bridge for the XGrammar library in swift/Sources/lib/CXGrammar has been extended to expose advanced guided-generation capabilities. Users can now roll back the grammar state by a specified number of tokens, retrieve the longest deterministic string for immediate output (jump-forward), and check if the grammar's root rule is fully completed. These features are implemented in the new xgrammar\_c\_bridge.cpp and updated xgrammar\_c\_bridge.h, replacing the previous header location and enabling more robust control over structured text generation.
swift/Sources · high confidence
Image segmenter CLI gains cache clearing and load timing
The image-segmenter tool now includes a --clear-coreai-cache flag that forces re-specialization of the Core AI model by clearing its cached specialization before loading. Additionally, the CLI now reports the model load time and indicates whether a cache hit occurred, helping users monitor performance. The implementation also adds macOS-specific guards for the file-opening helper and refactors bundle loading to use ImageSegmentationBundle instead of ModelBundle.
swift/Sources/Tools/image-segmenter · high confidence
Improved segmentation mask quality and centralized visualization logic
The segmentation postprocessor now uses bilinear upsampling instead of nearest-neighbor for both binary masks and probability grids, which eliminates staircase artifacts and produces smoother edges, particularly for low-resolution models like SAM3 lite. Additionally, visualization logic for heatmaps, instance colors, and HSV-to-RGB conversion has been moved to the shared \CoreAIShared\ module to reduce duplication.
swift/Sources/CoreAIImageSegmenter/Postprocessing · high confidence
Lazy model loading and agentic reasoning support
Model initialization now defaults to lazy loading, deferring the inference engine load until the first generation request, which reduces startup latency and memory usage; users can opt into eager loading via a new \LoadMode\ parameter. The system now supports agentic chain-of-thought formats, allowing models to emit reasoning content as structured messages rather than inline tags. Additionally, users can configure prefill chunk sizes and thresholds to optimize performance for specific hardware or model architectures.
swift/Sources/CoreAILanguageModels/LanguageModel · high confidence
Logger output redirected to stderr
The CLILogger now writes diagnostic messages to standard error instead of standard output. This ensures that stdout remains clean for programmatic output, such as JSONL results in replay mode, preventing log noise from interfering with structured data consumption.
swift/Sources/CoreAIShared/Logger · high confidence
Model exports default to release mode with optional debug info
The export scripts for CLAP, CLIP, Depth Anything, EDSR, Efficient SAM, PVT, RoBERTa, T5, and Wav2Vec2 now default to RELEASE mode when converting models to Core AI, producing smaller assets by embedding minimal debug information. Users can opt into DEBUG mode via the new --include-debug-info flag to embed full debug information for troubleshooting conversion issues. Additionally, the coreai-core and coreai-torch dependencies have been updated to versions 1.0.0b3 and 0.4.3 respectively, and the Efficient SAM export now defaults to 2 points per query to support box prompts.
(repo-wide) · high confidence
Object detector CLI now supports batch image processing and dynamic model inputs
The object detector command-line tool has been updated to accept multiple input images via the --image flag, enabling batch detection in a single run. When processing multiple images, the --output-image flag now expects a directory path where individual detection overlays are saved, and --output-json returns a structured array of results per image. Additionally, users can now override the model's input dimensions using --input-height and --input-width, which is required for dynamic Core AI models. A new --clear-coreai-cache flag allows forcing a re-specialization of the model by clearing the Core AI cache before loading.
swift/Sources/Tools/object-detector · high confidence
Refactored decoding strategies to use eager setup and async sequences
The decoding strategies in CoreAILanguageModels have been refactored to improve performance and API consistency. The \DecodingStrategy\ protocol now returns a typed \AsyncSequence\ (e.g., \VanillaDecodedSequence\, \ConstrainedDecodedSequence\) instead of a generic \AsyncThrowingStream\, with all session creation and tokenization performed eagerly before the sequence is returned. A new \PipelinedConstrainedDecodingStrategy\ has been added to enable GPU-accelerated grammar-constrained decoding by applying bitmasks directly in the MPSGraph sampler, avoiding CPU logit transfers. Additionally, \StopSequences\ now supports additional EOS token IDs from tokenizer configs, and log probability calculations utilize a new \LogProbabilities\ struct for better numerical stability.
swift/Sources/CoreAILanguageModels/DecodingStrategies · high confidence
Scheduler refactoring: removal of legacy schedulers and expansion of DiscreteFlowScheduler
The scheduler module has been significantly restructured. The PNDMScheduler and DPMSolverMultistepScheduler implementations have been removed, and the SchedulerType enum now only exposes discrete flow matching. The DiscreteFlowScheduler has been expanded to support Flux, Wan, and Sana Sprint models, introducing a new initializer for explicit sigma schedules (e.g., TrigFlow), a stochastic sampling step for consistency-distilled models, and a public currentSigma property to expose the denoised x0 estimate for live previews. Additionally, the PredictionType enum has been moved to the top-level Scheduler.swift file.
swift/Sources/CoreAIDiffusionPipeline/Schedulers · high confidence
Schema-constrained generation now limits whitespace and supports rollback/jump-forward
Guided JSON generation now prevents infinite whitespace loops by enforcing a default limit of 32 whitespace characters between tokens (configurable or disabled), which is applied via a new WhitespaceGuard that restricts the next-token mask. The ConstrainedGenerationSession also exposes rollback and jump-forward capabilities, allowing the generation engine to undo recent tokens and inject deterministic strings, while VocabularyType detection is improved to correctly identify byte-fallback tokenizers.
swift/Sources/CoreAILanguageModels/GuidedGeneration · high confidence
Streaming tool-call detection and grammar library cleanup
The CoreAILanguageModels module now includes a streaming parser (ToolCallParser) that detects tool-call blocks in the model's token stream, supporting JSON, ATEM, and Qwen3-Coder XML formats. A helper function (lastSafeIndex) ensures streaming markers are not split across deltas. Additionally, unused model shape configuration files and the bundled CXGrammar (XGrammar) framework headers have been removed from this location.
swift/Sources/CoreAILanguageModels · high confidence
Support for multi-function SAM3 Lite model assets
The image segmentation engine now automatically detects and supports two distinct model asset structures: the existing single-function models (like baseline SAM3 and EfficientSAM) and new multi-function models (produced by the SAM3 lite export). For multi-function assets, the engine dynamically wires together three separate graphs—image\_encode, text\_encode, and detect—at runtime to produce the same SegmentationOutput shape as before. This change allows users to leverage optimized SAM3 Lite models without code changes, while point-query support remains exclusive to single-function models.
swift/Sources/CoreAIImageSegmenter · high confidence
Support for new model kinds and improved bundle validation
The model bundle system now recognizes \videoDiffusion\, \videoSegmenter\, and \speechRecognizer\ kinds, enabling the runtime to handle video diffusion (Wan), SAM 3 video segmentation, and speech recognition models. The \LanguageBundle\ type has been renamed to \LanguageModelBundle\ to reflect its scope. Additionally, bundle loading now validates that the input is a directory rather than a direct model asset file (\.aimodel\/\.aimodelc\), providing a clear error message if a user points the runner at a single asset file. The asset verification method was also renamed to \verifyAssetsExisting\ for clarity, and error messages were updated to provide more specific guidance on missing assets and bundle structure.
swift/Sources/CoreAIShared/Bundle · high confidence
Vectorized log-probability computation and enhanced generation API
Text generation now includes a new LogProbabilities struct that uses the Accelerate framework for vectorized log-softmax calculations, significantly improving performance for per-token log probability and top-K alternative extraction. The TextGenerator API has been updated to return token IDs alongside text and logits in generateWithLogits, and a new evaluateRawTokens method allows direct evaluation of pre-tokenized token sequences. Additionally, the decoding strategy is now accessed asynchronously via try await to support serialization of back-to-back turns.
swift/Sources/CoreAILanguageModels/TextGeneration · high confidence
Whisper export defaults to release mode with optional debug info and dynamic shapes
The Whisper model export now defaults to release mode, producing smaller assets by embedding minimal debug information; users can opt into full debug info via the new --include-debug-info flag. Additionally, the export process now traces the model with dynamic shapes (specifically for decoder input sequence length), which improves flexibility for variable-length inputs during inference.
models/whisper · high confidence
Yolo export defaults to release mode with optional debug info and updated Core AI dependencies
The Yolo model export process now defaults to release mode, resulting in smaller exported assets, while adding a --include-debug-info flag to opt-in to debug information embedding. This change also updates the coreai-core and coreai-torch dependencies to versions 1.0.0b3 and 0.4.3 respectively. Additionally, the documentation reflects new capabilities for batched detection and dynamic shape handling in both iOS/macOS applications and the Mac command-line tool, including support for explicit input dimensions and kernel warmup.
models/yolo · high confidence
llm-runner adds video input, MinP sampling, repetition penalties, and VLM image options
The llm-runner tool now supports vision-language models with new command-line flags for video input (--video, --video-frames, --video-sampling) and configurable image preprocessing (--image, --image-strategy, --image-info). It introduces MinP sampling (--min-p) alongside existing Top-P, and adds repetition penalty controls (--repetition-penalty, --repetition-penalty-window) with validation against constrained generation. Prefill chunking is refined with a separate --chunk-threshold option and a default of 4096 tokens on macOS with \>=64GB memory. A new --clear-coreai-cache flag forces re-specialization of the Core AI model cache, and the internal bundle type has been renamed from LanguageBundle to LanguageModelBundle. Also in this area: 3 commits (0 fixes) modifying swift; 6 commits (1 fix) modifying swift; Inference options split into orthogonal token and logits axes; Unified stop-token resolution across text and VLM adapters; Core AI model asset validation now checks for valid model content, not just file existence.
swift/Sources/Tools/llm-runner · medium confidence · unverified
Fixes
Fixes token count mismatch error in logits output handling
The \LogitsWriter\ now accepts generated token IDs directly instead of decoding generated text to derive them, which resolves a token count mismatch error that occurred when the encoded token count differed from the logits count. This change ensures that the validation logic correctly compares the number of provided token IDs against the number of logits vectors, preventing crashes or invalid states during logits saving or console printing.
swift/Sources/CoreAILanguageModels/Output · high confidence
Test coverage
5 commits adding/updating tests in swift/Tests/GuidedGenerationTests; Added comprehensive test coverage for CoreAISpeech audio processing and configuration; Added test coverage for CoreAIShared utilities; Added test coverage for CoreAIVideoSegmenter components; Added test coverage for diffusion pipeline configuration, token math, and input validation; Added tests for ImageSegmentationBundle and updated segmentation engine tests; Added tests for ObjectDetector batch planning logic; Added tests for language model parsing, engine lifecycle, and configuration; Added tests for video input, reading, and writing in CoreAIShared; Added unit tests for CoreAILMCommon API types and server utilities; Removed unused XCTest plan file.
Dependencies
New video, speech, and LLM server capabilities with dependency updates
This release introduces new Swift libraries and tools for video diffusion (CoreAIVideoDiffusionPipeline), video segmentation (CoreAIVideoSegmenter), and speech recognition (CoreAISpeech), alongside a new llm-server executable for OpenAI-compatible LLM serving. The CXGrammar dependency is now a source-based target using XGrammar v0.2.2 instead of a binary framework. Python dependencies are updated, including transformers to v5.12.1, huggingface-hub to \>=1.5.0, and tokenizers capped at \<0.23, with new CLI scripts for VLM export and a flaky test marker added.
(dependencies) · high confidence
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
How this codebase got here
Score
- CAI 60 → 64 (+3.7)
- Rubric changed (rubric-2026.10.5 → rubric-2026.11.4) — scores are not directly comparable.
Lenses
- Code Health 66 → 76 (+10.4)
- Architecture 92 → 93 (+1.4)
- Maturity 58 → 58 (+0.2)
- Readiness 54 → 56 (+1.9)
- Security 75 → 83 (+8.7)
- Performance 83 → 100 (+17.1)
Resolved (62)
- Change-coupling hub: LLMRunnerMain.swift → CoreAISequentialVLMEngine.swift, CoreAIVisionLanguageModel.swift, LLMServerMain.swift (swift/Sources/Tools/llm-runner/LLMRunnerMain.swift)
- ClassTooLong: CoreAISequentialVLMEngine (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialVLMEngine.swift)
- CoreAISequentialVLMEngine.init (cognitive 27) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialVLMEngine.swift)
- CoreAISequentialVLMEngine.init (cyclomatic 21) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialVLMEngine.swift)
- Duplicated block (10 lines × 10) (models/clap/export.py)
- Duplicated block (10 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/RNG/NumPyRandomSource.swift)
- Duplicated block (11 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/Components/CoreAILatentCodec.swift)
- Duplicated block (11 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/RNG/NumPyRandomSource.swift)
- Duplicated block (12 lines × 2) (python/src/coreai_models/export/pipeline.py)
- Duplicated block (12 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/RNG/NumPyRandomSource.swift)
- Duplicated block (12 lines × 2) (swift/Sources/CoreAILanguageModels/Bundle/LanguageConfig.swift)
- Duplicated block (12 lines × 2) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAIPipelinedEngine.swift)
- Duplicated block (12 lines × 6) (swift/Sources/CoreAIDiffusionPipeline/Bundle/DiffusionConfig.swift)
- Duplicated block (12 lines × 7) (swift/Sources/CoreAIDiffusionPipeline/Bundle/DiffusionConfig.swift)
- Duplicated block (15–16 lines × 2) (swift/Sources/CoreAILanguageModels/TextGeneration/TextGenerator.swift)
- Duplicated block (17–18 lines × 2) (python/src/coreai_models/models/macos/muse_glimmer_drafter_dflash.py)
- Duplicated block (19 lines × 4) (models/clap/export.py)
- Duplicated block (19–20 lines × 3) (models/clap/export.py)
- Duplicated block (21–23 lines × 10) (models/clap/export.py)
- Duplicated block (22–23 lines × 7) (models/clip/export.py)
- …and 42 more
New (218)
- BaseForCausalLM.from_hf_memory_efficient (cognitive 16) (python/src/coreai_models/models/base.py)
- BatchScheduler.pump (cyclomatic 21) (swift/Sources/CoreAILanguageModels/InferenceEngines/BatchScheduler.swift)
- CPUDrafter.readSharedKV (cognitive 31) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/CPUDrafter.swift)
- CPUDrafter.readSharedKV (cyclomatic 18) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/CPUDrafter.swift)
- Change coupling: CoreAIVisionLanguageModel.swift ↔ LLMRunnerMain.swift (swift/Sources/CoreAILanguageModels/VLM/CoreAIVisionLanguageModel.swift)
- ChatHandler.swift.tokenizeMessages::render (cognitive 30) (swift/Sources/Tools/llm-server/ChatHandler.swift)
- ChatHandler.swift.tokenizeMessages::render (cyclomatic 16) (swift/Sources/Tools/llm-server/ChatHandler.swift)
- CoreAISequentialEngine.generationResetPlan (cognitive 16) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAISequentialEngine.swift)
- DiffusionRunner.resolveOutputSize (cognitive 17) (swift/Sources/Tools/diffusion-runner/DiffusionRunnerMain.swift)
- DraftPack.require (cognitive 19) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
- DraftPack.require (cyclomatic 17) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
- DraftPack.validate (cognitive 29) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
- DraftPack.validate (cyclomatic 20) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/DraftScan.swift)
- Duplicated block (10 lines × 2) (swift/Sources/CoreAIShared/RNG/NumPyRandomSource.swift)
- Duplicated block (11 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/Components/CoreAILatentCodec.swift)
- Duplicated block (12 lines × 2) (swift/Sources/CoreAIDiffusionPipeline/Pipelines/FlowTransformerPipeline+Resources.swift)
- Duplicated block (12 lines × 2) (swift/Sources/CoreAILanguageModels/InferenceEngines/CoreAIPipelinedEngine.swift)
- Duplicated block (15–16 lines × 2) (swift/Sources/CoreAILanguageModels/SpeculativeDecoding/CPUDrafter.swift)
- Duplicated block (19 lines × 3) (models/parakeet/export.py)
- Duplicated block (21–23 lines × 8) (models/depth-anything/export.py)
- …and 198 more
Changes since last survey
- 60 commits — 55 feature/other, 5 fixes
By area
- swift/Sources — 31 commits
- python/src — 12 commits
- python/tests — 3 commits
- (root) — 2 commits
- .github/workflows — 2 commits
- models/README.md — 2 commits
- models/gemma4 — 2 commits
- skills/skills — 2 commits
- .github/scripts — 1 commit
- models/efficient-sam — 1 commit
- models/parakeet — 1 commit
- swift/Tests — 1 commit
Notable commits
- fix: Fix graph mode quantization for gemma3n models (#407)
- fix: Fix graph mode quantization for qwen3_5 models (#389)
- fix: Fix slow logits top-k extraction in LogitsWriter (#512)
- fix: Fix the segmenter module name in the README samples (#488)
- fix: Fix two stale snippets in the skills docs (#431)
- change: Run several prompts at once in llm-runner and llm-benchmark, and refuse batched requests past the device's memory recommendation (#511)
- change: quantize on a chosen device and externalize composites for every backend (#408)
- change: Add EmbeddingGemma 2 export: coreai.embedding.export and registry entry (#369)
- change: Add native quantized weight modules for Gemma 4 (#419)
- change: Add the static-shape draft head and the engine hooks it reads (#394)
- change: Consolidate random number generators into CoreAIShared (#273) (#421)
- change: Constrained session tests go through ConstrainedSessionSlot (#526)
- change: DiffusionGemma: keep RoPE inv_freq buffer in float32 under to(dtype) (#482) (#483)
- change: Document speculative decoding in llm-runner (#403)
- change: Document the real engine variants (#489)
- change: Draft head: start reading the scan table in at load (#410)
- change: Dynamic Flux2 (#510)
- change: Embedding modality: CLIP and CLAP bundles through coreai.embedding.export (#395)
- change: Fold the VLM engine into CoreAISequentialEngine (#388)
- change: Follow-ups to the VLM fold (#416)
- …and 40 more
Architecture
- Unchanged — 0 containers · 2 contexts · 0 edges
Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.
Survey your own repository
apple/coreai-models was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.
About this page
- The score is its most recent published measurement, taken on 15 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
- Measured at commit 128d2f30c935d27807d0e41a0f22a985f5d38250 — the exact code this score is about.
- Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
- Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.65+preprod-3c33c1882dc1.