Tag: verilog
-
Train a Transformer on Silicon: #2 A Decoder Is Just an Encoder with a Mask
v0 is done: a nanoGPT-faithful decoder layer that trains itself — forward, backprop, and SGD update — bit-exact against a golden model and taken to a clean 45 nm layout. The compute fit in 0.117 mm² of logic; the memory took 85% of the die.
-
Train a Transformer on Silicon: #1 Backprop on a Chip
Most AI chips run a model someone else trained; this one does the training itself — forward pass, backpropagation, and weight update, in fixed-point RTL, all the way down to a 45 nm layout.