karpathy/nanochat

A new \scripts/infer\_bench.py\ script is introduced to measure inference latency, throughput, memory and bandwidth utilization of a trained checkpoint. It sweeps over the decode batch size to trace the latency-throughput tradeoff between the compute-bound prefill and memory-bandwidth-bound decode regimes, reporting metrics like TTFT, MFU, and MBU.

scripts · high confidence

The codebase now includes a unified Flash Attention 3 interface that automatically falls back to PyTorch's SDPA on incompatible CUDA GPUs, MPS, and CPU. This new module, \nanochat/flash\_attention.py\, provides a drop-in replacement for the previous Flash Attention 3 implementation, supporting both training (no KV cache) and inference (with KV cache) modes. The implementation detects available hardware and selects the appropriate backend, ensuring compatibility across different GPU architectures and non-CUDA devices.

nanochat · high confidence

The embedded rustbpe Rust source code (lib.rs) and its README have been deleted from the repository. This change reflects the decision to treat rustbpe as an external dependency rather than maintaining the code inline, aligning with the project's move to host rustbpe as a separate repository.

rustbpe · high confidence

The tasks module no longer depends on the external \datasets\ library. Instead, it uses a new \load\_hub\_dataset\ function in \tasks/common.py\ that downloads and reads Hugging Face datasets directly from their auto-generated Parquet shards using \pyarrow\. This change also updates the MMLU task to use \auxiliary\_train\ as a split name rather than a subset, and corrects the MMLU subset assertion to only allow \all\ while moving \auxiliary\_train\ to the split argument.

tasks · high confidence

The \speedrun.sh\ script has been moved to the \runs/\ directory and updated to train a deeper (d24) model with a lower data-to-parameter ratio (8 instead of 10.5) to improve training efficiency. The script now uses \--fp8\ precision and a device batch size of 16. The previous midtraining phase has been replaced with Supervised Fine-Tuning (SFT). Additionally, new scripts \miniseries.sh\ and \scaling\_laws.sh\ have been added to the \runs/\ directory to support automated training series and scaling law analysis, while the \runcpu.sh\ script has been added for CPU-based educational runs.

runs · high confidence

The pretraining dataset has been upgraded from FineWeb-EDU to NVIDIA ClimbMix, a 400B-token mixture of high-quality web text, code, and math. This change reduces the time to reach GPT-2 capability from approximately 2 hours 46 minutes to 2 hours 1 minute. The update includes a migration warning in the data loading logic to guide users to the new dataset, and the reference script for data preparation now supports both the legacy FineWeb-EDU and the new ClimbMix configurations.

dev · high confidence

Added new test files to verify the correctness of the Flash Attention 3 and SDPA fallback implementations, the Engine class, the MuonAdamW optimizer, the Task/HubDataset machinery, the RustBPE tokenizer, and the sandboxed code execution environment. These tests ensure that attention mechanisms produce identical results across backends, that multi-sample generation samples tokens independently, that the optimizer matches PyTorch's AdamW behavior and converges, that task slicing and shuffling are deterministic, that the tokenizer handles special tokens and conversation rendering correctly, and that the execution sandbox properly restricts dangerous operations.

tests · high confidence

The project has removed the inline Rust-based \rustbpe\ crate and its associated \Cargo.lock\ and \Cargo.toml\ files, replacing them with an external \rustbpe\ PyPI dependency. Additionally, the \pyproject.toml\ has been updated to support both CPU and GPU installations via optional dependencies (\cpu\ and \gpu\), allowing users to install PyTorch for either platform. Unnecessary dependencies like \datasets\, \fastapi\, \uvicorn\, and \tokenizers\ have been removed, while \pyarrow\ and \filelock\ have been added.

(dependencies) · medium confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 46 → 51 (+5.3)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 97 → 94 (-3.6)
• Architecture 94 → 100 (+5.9)
• Maturity 57 → 62 (+5.5)
• Readiness 17 → 25 (+8.3)
• Security 81 → 79 (-1.2)

• Coverage not measured — test suite did not build
• Dimension evaluation failed
• Duplicated block (6 lines × 2) (tests/test_attention_fallback.py)
• Low: security finding (details withheld)
• Low: security finding (details withheld)
• Medium: security finding (details withheld)
• Medium: security finding (details withheld)
• No exposed public API
• No tests found
• Test reliability not included

• Critical CVE: [GHSA redacted] (uv.lock)
• Duplicated block (6–8 lines × 2) (nanochat/flash_attention.py)
• Duplicated block (7 lines × 2) (scripts/chat_eval.py)
• Engine.generate (cognitive 33) (nanochat/engine.py)
• Engine.generate (cyclomatic 18) (nanochat/engine.py)
• Further sole-owners (lower concentration)
• HackComment (nanochat/checkpoint_manager.py)
• High CVE: [GHSA redacted] (uv.lock)
• High CVE: [GHSA redacted] (uv.lock)
• High CVE: [GHSA redacted] (uv.lock)
• High CVE: [GHSA redacted] (uv.lock)
• High CVE: [GHSA redacted] (uv.lock)
• Hotspot: scripts/chat_sft.py (scripts/chat_sft.py)
• Low: security finding (details withheld)
• Low: security finding (details withheld)
• Medium CVE: [GHSA redacted] (uv.lock)
• Medium CVE: [GHSA redacted] (uv.lock)
• Medium CVE: [GHSA redacted] (uv.lock)
• Medium CVE: [GHSA redacted] (uv.lock)
• Medium CVE: [GHSA redacted] (uv.lock)
• …and 29 more

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

karpathy/nanochat was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.