Aleksandra Bakalova

Saarbrücken, Germany

profile-picture-me.jpg

I am Sasha (short for Aleksandra) Bakalova, PhD student in LaCoCo Lab at Saarland University supervised by Prof. Michael Hahn. I am also a member of RTG ‘‘Neuroexplicit Models of Language, Vision, and Action’’.

I did my Masters in Language Science and Technology at Saarland University, and my undergraduate in Applied Mathematics and Computer Science at HSE University. I learned a lot about deep learning at Yandex School of Data Analysis.

My research interests lie in the direction of interpretability, deep learning theory, and understanding what makes modern deep learning models so powerful. I’d be excited to work on anything that brings us closer to understanding what hides behind those matrix multiplications, how we can use it to build better models and use existing models in better ways.

If you wish to chat, feel free to drop me an email!

News

Nov 01, 2025 Excited to start a new chapter as a PhD student at the RTG ‘‘Neuroexplicit Models of Language, Vision, and Action’’ under the supervision of Prof. Michael Hahn!
Jul 31, 2025 Attended EEML 2025, got a best poster award! Walking around Sarajevo, getting inspired by the amazing people there and their work: it was a lot of fun.
Jul 07, 2025 Our work on understanding in-context learning is accepted to COLM 2025!
May 30, 2025 Born a Transformer – Always a Transformer?: new preprint is on arxiv! We try to bring theory closer to practice and answer a question: how do theoretical limitations of transformers manifest in pre-trained models? See Yana’s X post for a short explanation.
Mar 31, 2025 Our preprint on understanding in-context learning in Gemma2-2b is on arxiv! We found a circuit in the model that performs in-context learning and interpreted the information flow in it. See this X post for a short explanation.
Aug 01, 2024 Started my Erasmus semester at Charles University in Prague. In love with the city!
Apr 30, 2024 Attended ALPS 2024 for some inspiration and breathtaking views :)
Oct 30, 2023 Started a job as a Research Assistant at LaCoCo Lab with Dr. Michael Hahn. Very excited to work on LLM interpretability with such amazing colleagues!
Oct 01, 2023 Started my Masters in Language Science and Technology at Saarland University.

Publications

2025

  1. Born a Transformer–Always a Transformer?
    Yana Veitsman, Mayank Jobanputra, Yash Sarrof, Aleksandra Bakalova, Vera Demberg, Ellie Pavlick, and Michael Hahn
    arXiv preprint arXiv:2505.21785, 2025
  2. Contextualize-then-Aggregate: Circuits for In-Context Learning in Gemma-2 2B
    Aleksandra Bakalova, Yana Veitsman, Xinting Huang, and Michael Hahn
    Second Conference on Language Modeling, 2025

2024

  1. LUNA: A framework for language understanding and naturalness assessment
    Marat Saidov, Aleksandra Bakalova, Ekaterina Taktasheva, Vladislav Mikhailov, and Ekaterina Artemova
    arXiv preprint arXiv:2401.04522, 2024