karpathy/nanoGPT

A new character-level version of the Shakespeare dataset is now available for training character-level language models. This includes a \prepare.py\ script that downloads the raw text, maps each character to an integer, and saves the resulting token arrays to \train.bin\ and \val.bin\ files, along with a \meta.pkl\ containing the character-to-integer mappings. A \readme.md\ has been added to explain the dataset's structure and the output of the preparation script.

_data/shakespeare\char · high confidence

A new \prepare.py\ script has been added to the \data/shakespeare\ directory to download the tiny Shakespeare dataset, encode it using the GPT-2 tokenizer, and export the training and validation data as binary files. A corresponding \readme.md\ has been added to document the dataset and the output file sizes.

data/shakespeare · high confidence

New configuration files have been added to support evaluating and training GPT-2 models. This includes evaluation configs for the base, medium, large, and XL variants of GPT-2, as well as training configurations for fine-tuning on the Shakespeare dataset and training a character-level model on Shakespeare. These files define hyperparameters such as batch size, learning rate, and model initialization settings for these specific use cases.

config · high confidence

The model now supports optional bias in LayerNorm and Linear layers, with a default of True to match GPT-2 architecture. The implementation adds support for PyTorch's scaled\_dot\_product\_attention (Flash Attention) when available, falling back to a manual causal mask otherwise. Additionally, the model can be compiled using torch.compile for significant throughput improvements. The vocabulary size is padded to 50304 for efficiency, and the configurator script allows command-line overrides for all hyperparameters.

(repo-wide) · high confidence

The data preparation script for OpenWebText has been refactored to improve performance and reliability. The script now uses relative paths to ensure files are created in the local folder regardless of the working directory. It also switches from using \mmap\ and \subprocess\ for file writing to using \numpy.memmap\ and batched writes with \tqdm\ for progress tracking. Additionally, the number of processes for loading the dataset is now configurable via \num\_proc\_load\dataset\, and the script is wrapped in an \if \\name\\_ == '\_\main\\_':\ block to support multiprocessing on all platforms, including macOS.

data/openwebtext · medium confidence

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

• CAI 41 → 35 (-6.1)
• Rubric changed (rubric-2026.08.15 → rubric-2026.09.15) — scores are not directly comparable.

• Code Health 100 → 99 (-0.7)
• Architecture 69 → 69 (+0.0)
• Maturity 51 → 30 (-20.8)
• Readiness 17 → 15 (-1.4)
• Security 71 → 71 (+0.0)

• Dimension evaluation failed
• No exposed public API

• Change coupling: bench.py ↔ train.py (bench.py)
• Change coupling: sample.py ↔ train.py (sample.py)
• Orphaned files with no living knowledge
• TodoComment (model.py)
• TodoComment (sample.py)

Written by watchdog.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

karpathy/nanoGPT was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.