Tag: language-model

The chip could train, but it had nothing to decode. This design adds token and positional embeddings, an LM head, softmax, and cross-entropy on top of the packed memory map from #3 — and the toy copy-task becomes a real, if tiny, language model that overfits a next-token sequence on-chip. Cross-entropy falls 4.144 to 0.0149…