New project and vacancy

A new project in our team

A funding application that I co-authored together with my colleague Rebecka Jörnsten has been approved for funding within the Wallenberg AI, Autonomous Systems and Software Program WASP. The application is entitled “Understanding transformers via compression and discrimination”.

Understanding transformers via information theory

Transformers have rapidly become the leading architecture in deep learning, powering state-of-the-art models across natural language processing, vision, and beyond. Despite their success, transformers are built on empirical foundations and remain poorly understood from a theoretical perspective. Their inner workings, limitations, and vulnerabilities are still largely opaque. This poses fundamental challenges to trustworthy and robust AI.

In this project, we aim to address this gap by providing a rigorous mathematical understanding of transformers through the lens of information theory. In particular, the project will explore the concepts of sufficiency, i.e., how effectively a transformer captures predictive features, and of minimality, i.e., how compact and discriminative those features are.

A fully funded PhD position

As part of this project, we have a fully funded PhD student position. The selected candidate will also join WASP graduate school, which offers unparalleled resources as well as fantastic networking and career opportunities. If you are interested, consider applying by following the instructions described in Chalmers vacancy announcement. Please note that the applications should be submitted using Chalmers application portal; applications received per email will not be considered.