Skip to content
CAI
Software that uses CAICheck a score

lucidrains/vit-pytorch

51.5

Adequate · 16 November 2026

15.6k

lines of production code

Python

primary language

5

measurements over time

CAI band scale
CAI trend line
CAI lens gauges

What this system is

Features

Added training script for ViT with decorrelation auxiliary loss

Users can now train Vision Transformers using a decorrelation auxiliary loss to regularize multi-head attention and feedforward subspaces. This change introduces a new \ViT\ variant (\vit\_pytorch.vit\_with\_decorr\) and a dedicated training script (\train\_vit\_decorr.py\) that demonstrates how to apply this regularization on CIFAR-100 using Accelerate and Weights & Biases.

(repo-wide) · high confidence

New Vision Transformer architectures and self-supervised learning wrappers

This release adds several new Vision Transformer variants and self-supervised learning frameworks to the library. New model implementations include CaiT (with LayerScale and cross-attention), CrossViT (multi-scale encoder with cross-attention), CrossFormer (dynamic positional bias), CvT (convolutional token mixing), DeepViT (re-attention), CCT (compact conv-based transformer), and ATS-ViT (adaptive token sampling). Additionally, wrappers and implementations for self-supervised techniques are introduced, including DINO (with EMA teacher and distillation loss), MAE (Masked Autoencoder), and Masked Position Prediction (MPP). A new AcceptVideoWrapper allows wrapping existing image networks for video processing with optional time positional embeddings and MOSS integration.

_vit\pytorch · high confidence

New example notebook for training a Visual Transformer on Cats vs Dogs data

Added a new Jupyter notebook example (\examples/cats\_and\_dogs.ipynb\) that demonstrates how to train a Visual Transformer using the Linformer efficient attention implementation on the Kaggle Dogs vs Cats dataset. The notebook includes setup for dependencies (\vit\_pytorch\, \linformer\), data loading, and model training configuration.

examples · high confidence

Test coverage

Added tests for DINO loss stability and ViT model functionality

Added new test coverage for the DINO distillation loss function, specifically verifying that loss values and gradients remain finite and numerically stable when using low-precision data types (float16 and bfloat16), and confirming gradient correctness against a reference implementation. Additionally, added a basic functional test for the ViT model to ensure it produces correctly shaped output logits.

tests · high confidence

Dependencies

vit-pytorch version 1.26.7 release

The package has been updated to version 1.26.7. The build system now requires setuptools\>=61 and wheel, and the project mandates Python 3.8 or higher. Runtime dependencies are specified as einops\>=0.8.2, torch\>=2.5, and torchvision. Test dependencies include pytest and specific pinned versions of torch (2.5.0) and torchvision (0.20.0).

(dependencies) · high confidence

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

How this codebase got here

Score

  • CAI 52 → 52 (-0.4)
  • Rubric changed (rubric-2026.11.2 → rubric-2026.11.4) — scores are not directly comparable.

Lenses

  • Code Health 67 → 57 (-10.5)
  • Architecture 86 → 100 (+14.2)
  • Maturity 50 → 50 (+0.0)
  • Readiness 38 → 38 (+0.0)
  • Security 100 → 100 (+0.0)

Resolved (4)

  • Duplicated block (15 lines × 2) (vit_pytorch/vit_nd_pope.py)
  • Duplicated block (19 lines × 2) (vit_pytorch/vaat.py)
  • Duplicated block (7 lines × 3) (vit_pytorch/vaat.py)
  • Projects may be oversized for their cohesion

New (73)

  • Edited copies of one member (3 members, 10+ corresponding lines) (vit_pytorch/local_vit.py)
  • Edited copies of one member (3 members, 10+ corresponding lines) (vit_pytorch/simple_uvit.py)
  • Edited copies of one member (3 members, 13+ corresponding lines) (vit_pytorch/levjepa_nd.py)
  • Edited copies of one member (3 members, 15+ corresponding lines) (vit_pytorch/cvt.py)
  • Edited copies of one member (3 members, 15+ corresponding lines) (vit_pytorch/deepvit.py)
  • Edited copies of one member (3 members, 17+ corresponding lines) (vit_pytorch/mp3.py)
  • Edited copies of one member (3 members, 18+ corresponding lines) (vit_pytorch/vit.py)
  • Edited copies of one member (3 members, 23+ corresponding lines) (vit_pytorch/cait.py)
  • Edited copies of one member (3 members, 3+ corresponding lines) (vit_pytorch/dino.py)
  • Edited copies of one member (3 members, 31+ corresponding lines) (vit_pytorch/jet_vit.py)
  • Edited copies of one member (3 members, 4+ corresponding lines) (vit_pytorch/dino.py)
  • Edited copies of one member (3 members, 4+ corresponding lines) (vit_pytorch/levjepa.py)
  • Edited copies of one member (3 members, 58+ corresponding lines) (vit_pytorch/vit_nd.py)
  • Edited copies of one member (4 members, 2+ corresponding lines) (vit_pytorch/regionvit.py)
  • Edited copies of one member (4 members, 4+ corresponding lines) (vit_pytorch/levjepa.py)
  • Edited copies of one member (4 members, 4+ corresponding lines) (vit_pytorch/looped_vit.py)
  • Edited copies of one member (4 members, 8+ corresponding lines) (vit_pytorch/cait.py)
  • Edited copies of one member (5 members, 25+ corresponding lines) (vit_pytorch/deepvit.py)
  • Edited copies of one member (6 members, 25+ corresponding lines) (vit_pytorch/ats_vit.py)
  • Edited copies of one member (7 members, 14+ corresponding lines) (vit_pytorch/cross_vit.py)
  • …and 53 more

Changes since last survey

  • 1 commits — 1 feature/other, 0 fixes

By area

  • (root) — 1 commit

Notable commits

  • change: turns out a group out there thought of the multihead attention residual idea concurrently

Written by cca.canine.dev from the codebase's own history, inside the signed delivery this page is composed from.

Survey your own repository

lucidrains/vit-pytorch was measured the same way every project in this corpus was: the same rubric, at a pinned commit, with the result published in full. Point a surveyor at a repository you know and see whether you agree with it.

About this page

  • The score is its most recent published measurement, taken on 16 November 2026 at a pinned commit. It is not a live figure and does not change until the project is measured again.
  • Measured at commit 42298e16ad7915ea6f531049e7909e7ee07bc5fe — the exact code this score is about.
  • Scored under rubric-2026.11.4 — the same rubric and the same method as every other entry in this index.
  • Measured by Canine Code Analyzer (cca.canine.dev) using cca measure-2026.11.73+preprod-52de90b74216.