A hand-written implementation of the GPT-Neo transformer forward pass in plain TypeScript with zero runtime dependencies, bundled with the TinyStories-1M checkpoint (7.5 MB, fp16 safetensors). It exposes every intermediate of the computation — token and position embeddings, the residual stream after each block, per-layer per-head attention weights and next-token logits — and is verified token-exact against the reference Hugging Face implementation (greedy continuation) with logits and attention maps reproduced within fp16 tolerance. Includes cosine nearest-neighbour search over the embedding table, automatic discovery of interpretable attenti
A long-form interactive essay in which the reader types their own sentence and watches a real language model, running entirely in the browser, tokenize it, embed it, attend over it, and sample the next token. Includes a pure TypeScript implementation of the GPT-Neo forward pass (TinyStories-1M, verified token-exact against the reference implementation), a live attention explorer with automatic feature-head discovery, and a bilingual (EN/中文) narrative aimed at non-engineers.
