nano-lm: an auditable GPT-Neo forward pass in dependency-free TypeScript
A hand-written implementation of the GPT-Neo transformer forward pass in plain TypeScript with zero runtime dependencies, bundled with the TinyStories-1M checkpoint (7.5 MB, fp16 safetensors). It exposes every intermediate of the computation — token and position embeddings, the residual stream after each block, per-layer per-head attention weights and next-token logits — and is verified token-exact against the reference Hugging Face implementation (greedy continuation) with logits and attention maps reproduced within fp16 tolerance. Includes cosine nearest-neighbour search over the embedding table, automatic discovery of interpretable attenti
Resource information
- Format
- zip
- Original repository
- 22089054
- DOI
- 10.5281/zenodo.22089054
- Credit basis
- Creators in Zenodo record metadata
- Access
- Open files available from the original Zenodo deposit
- Last checked
- Nov 5, 2026
- License
- mit-license
- Last record update
- Nov 5, 2026
About this resource
A hand-written implementation of the GPT-Neo transformer forward pass in plain TypeScript with zero runtime dependencies, bundled with the TinyStories-1M checkpoint (7.5 MB, fp16 safetensors). It exposes every intermediate of the computation — token and position embeddings, the residual stream after each block, per-layer per-head attention weights and next-token logits — and is verified token-exact against the reference Hugging Face implementation (greedy continuation) with logits and attention maps reproduced within fp16 tolerance. Includes cosine nearest-neighbour search over the embedding table, automatic discovery of interpretable attention heads (previous-token, first-token anchor, most diffuse), and top-k / temperature sampling utilities. Extracted from the interactive essay "Inside the Machine", where it powers the in-browser dissection widgets.