Category: Reading
-
Weight-sparse transformers have interpretable circuits
Weight-sparse transformers have interpretable circuits1234: Train sparse model on weights and pruning to explain interpretability, find connections between sparse and dense models. Transform: Encoder, Decoder, from tokens to embeddings to tokens | from electricity to magnetics to electricity | Fourier Transform | LLM Visualization Overall Setup | Superposition 5 Sparse Model Training sparse models contain…
-
Research Papers Manifestation
As a researcher myself, reading papers are one part of the job. Although there are some resources and materials present ways to read papers, sometimes I still don’t feel that they are pracitical and cannot help me digest the papers properly. Before going further, I need to recommend the book ” How to read a…