Announcement Born Paper

Born a Transformer – Always a Transformer?: new preprint is on arxiv! We try to bring theory closer to practice and answer a question: how do theoretical limitations of transformers manifest in pre-trained models? See Yana’s X post for a short explanation.