2026
ScratchGPT
Implemented a character-level decoder-only Transformer in PyTorch from first principles, including the data pipeline, causal self-attention, multi-head attention, residual blocks, evaluation, checkpointing and autoregressive generation.
View on GitLab