Division of Labour in Speech Foundation Models: Encoder Representations and Decoder Predictions

Le projet « Division of Labour in Speech Foundation Models: Encoder Representations and Decoder Predictions »,  porté par Guillaume Wisniewski, est lauréat de l’APP 2026 Université Paris Cité – Université de Toronto.  Il est porté à Toronto par Ewan Dunbar et à Université Paris Cité par le LLF avec la collaboration d’ALTAE et est financé dans le cadre des projets IDEX. L’objectif du projet est de mieux comprendre le fonctionnement des grands modèles de la parole.

Recent advances in neural audio models have led to major progress in automatic speech recognition (the task of converting spoken language into text) making it reliable and widely usable across many real-world applications. Despite the diversity of existing approaches, these systems share a key limitation: they operate largely as black boxes, learning speech representations directly from data without explicit linguistic supervision. This raises important scientific challenges, in particular how to interpret and understand the structure of these learned representations. At Université Paris Cité and University of Toronto, two research groups working at the intersection of artificial intelligence, computer science and linguistics have developed complementary expertise, focusing on different model architectures and analytical methods. This project aims to bring these teams together to confront and combine their approaches, with the goal of providing a more comprehensive understanding of modern speech systems and fostering the development of new methods for analysing them. The collaboration will be structured around regular joint seminars, two workshops (an opening meeting in September 2026 and a closing event in February 2028), and reciprocal research visits, including extended stays by graduate students at both institutions.

16h-20h KICK OFF MEETING Wednesday 2nd of September 16h-20h CET (= 10:00 AM to 2:00 PM Toronto time)
https://u-paris.zoom.us/my/nballier?pwd=b2dsYWRuTTdnT0RpUUpEQzZLQ2ovQT09

Personnes d’ALTAE participant au projet : Nicolas Ballier (PU), Lichao Zhu, Maria Zimina-Poirot et Behnoosh Namdarzadeh.