-
Teaching and learning in the age of artificial intelligence
Authors:
Margarida Romero,
Laurent Heiser,
Alexandre Lepage,
Alexandre Lepage,
Anne Gagnebien,
Audrey Bonjour,
Aurélie Lagarrigue,
Axel Palaude,
Caroline Boulord,
Charles-Antoine Gagneur,
Chloé Mercier,
Christelle Caucheteux,
Dominique Guidoni-Stoltz,
Florence Tressols,
Frédéric Alexandre,
Jean-François Céci,
Jean-François Metral,
Jérémy Camponovo,
Julie Henry,
Laurent Fouché,
Laurent Heiser,
Lianne-Blue Hodgkins,
Margarida Romero,
Marie-Hélène Comte,
Michel Durampart
, et al. (10 additional authors not shown)
Abstract:
As part of the Digital Working Group (GTnum) #Scol_IA "Renewal of digital practices and creative uses of digital and AI" we are pleased to present the white paper "Teaching and learning in the era of Artificial Intelligence, Acculturation, integration and creative uses of AI in education". The white paper edited by Margarida Romero, Laurent Heiser and Alexandre Lepage aims to provide the various e…
▽ More
As part of the Digital Working Group (GTnum) #Scol_IA "Renewal of digital practices and creative uses of digital and AI" we are pleased to present the white paper "Teaching and learning in the era of Artificial Intelligence, Acculturation, integration and creative uses of AI in education". The white paper edited by Margarida Romero, Laurent Heiser and Alexandre Lepage aims to provide the various educational actors with a diversified perspective both on the issues of acculturation and training in AI and on the resources and feedback from the various research teams and organisations. of scientific culture in the French-speaking countries. A multidisciplinary approach makes it possible to consider the perspectives of researchers in computer science as well as those of education and training sciences, information and communication sciences and the expertise of teaching professionals. and scientific mediation.
△ Less
Submitted 14 March, 2023; v1 submitted 13 March, 2023;
originally announced March 2023.
-
Decoding speech perception from non-invasive brain recordings
Authors:
Alexandre Défossez,
Charlotte Caucheteux,
Jérémy Rapin,
Ori Kabeli,
Jean-Rémi King
Abstract:
Decoding speech from brain activity is a long-awaited goal in both healthcare and neuroscience. Invasive devices have recently led to major milestones in that regard: deep learning algorithms trained on intracranial recordings now start to decode elementary linguistic features (e.g. letters, words, spectrograms). However, extending this approach to natural speech and non-invasive brain recordings…
▽ More
Decoding speech from brain activity is a long-awaited goal in both healthcare and neuroscience. Invasive devices have recently led to major milestones in that regard: deep learning algorithms trained on intracranial recordings now start to decode elementary linguistic features (e.g. letters, words, spectrograms). However, extending this approach to natural speech and non-invasive brain recordings remains a major challenge. Here, we introduce a model trained with contrastive-learning to decode self-supervised representations of perceived speech from the non-invasive recordings of a large cohort of healthy individuals. To evaluate this approach, we curate and integrate four public datasets, encompassing 175 volunteers recorded with magneto- or electro-encephalography (M/EEG), while they listened to short stories and isolated sentences. The results show that our model can identify, from 3 seconds of MEG signals, the corresponding speech segment with up to 41% accuracy out of more than 1,000 distinct possibilities on average across participants, and more than 80% in the very best participants - a performance that allows the decoding of words and phrases absent from the training set. The comparison of our model to a variety of baselines highlights the importance of (i) a contrastive objective, (ii) pretrained representations of speech and (iii) a common convolutional architecture simultaneously trained across multiple participants. Finally, the analysis of the decoder's predictions suggests that they primarily depend on lexical and contextual semantic representations. Overall, this effective decoding of perceived speech from non-invasive recordings delineates a promising path to decode language from brain activity, without putting patients at risk for brain surgery.
△ Less
Submitted 5 October, 2023; v1 submitted 25 August, 2022;
originally announced August 2022.
-
Toward a realistic model of speech processing in the brain with self-supervised learning
Authors:
Juliette Millet,
Charlotte Caucheteux,
Pierre Orhan,
Yves Boubenec,
Alexandre Gramfort,
Ewan Dunbar,
Christophe Pallier,
Jean-Remi King
Abstract:
Several deep neural networks have recently been shown to generate activations similar to those of the brain in response to the same input. These algorithms, however, remain largely implausible: they require (1) extraordinarily large amounts of data, (2) unobtainable supervised labels, (3) textual rather than raw sensory input, and / or (4) implausibly large memory (e.g. thousands of contextual wor…
▽ More
Several deep neural networks have recently been shown to generate activations similar to those of the brain in response to the same input. These algorithms, however, remain largely implausible: they require (1) extraordinarily large amounts of data, (2) unobtainable supervised labels, (3) textual rather than raw sensory input, and / or (4) implausibly large memory (e.g. thousands of contextual words). These elements highlight the need to identify algorithms that, under these limitations, would suffice to account for both behavioral and brain responses. Focusing on the issue of speech processing, we here hypothesize that self-supervised algorithms trained on the raw waveform constitute a promising candidate. Specifically, we compare a recent self-supervised architecture, Wav2Vec 2.0, to the brain activity of 412 English, French, and Mandarin individuals recorded with functional Magnetic Resonance Imaging (fMRI), while they listened to ~1h of audio books. Our results are four-fold. First, we show that this algorithm learns brain-like representations with as little as 600 hours of unlabelled speech -- a quantity comparable to what infants can be exposed to during language acquisition. Second, its functional hierarchy aligns with the cortical hierarchy of speech processing. Third, different training regimes reveal a functional specialization akin to the cortex: Wav2Vec 2.0 learns sound-generic, speech-specific and language-specific representations similar to those of the prefrontal and temporal cortices. Fourth, we confirm the similarity of this specialization with the behavior of 386 additional participants. These elements, resulting from the largest neuroimaging benchmark to date, show how self-supervised learning can account for a rich organization of speech processing in the brain, and thus delineate a path to identify the laws of language acquisition which shape the human brain.
△ Less
Submitted 20 March, 2023; v1 submitted 3 June, 2022;
originally announced June 2022.
-
Long-range and hierarchical language predictions in brains and algorithms
Authors:
Charlotte Caucheteux,
Alexandre Gramfort,
Jean-Remi King
Abstract:
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchi…
▽ More
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
△ Less
Submitted 28 November, 2021;
originally announced November 2021.
-
Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects
Authors:
Charlotte Caucheteux,
Alexandre Gramfort,
Jean-Rémi King
Abstract:
A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli (e.g. regular speech versus scrambled words, sentences, or paragraphs). Although successful, this `model-free' approach necessitates the acquisition of a large and costly set of neuroimaging data. Here, we show that a model-based approach can reach equi…
▽ More
A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli (e.g. regular speech versus scrambled words, sentences, or paragraphs). Although successful, this `model-free' approach necessitates the acquisition of a large and costly set of neuroimaging data. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli. We capitalize on the recently-discovered similarities between deep language models and the human brain to compute the mapping between i) the brain responses to regular speech and ii) the activations of deep language models elicited by modified stimuli (e.g. scrambled words, sentences, or paragraphs). Our model-based approach successfully replicates the seminal study of Lerner et al. (2011), which revealed the hierarchy of language areas by comparing the functional-magnetic resonance imaging (fMRI) of seven subjects listening to 7min of both regular and scrambled narratives. We further extend and precise these results to the brain signals of 305 individuals listening to 4.1 hours of narrated stories. Overall, this study paves the way for efficient and flexible analyses of the brain bases of language.
△ Less
Submitted 12 October, 2021;
originally announced October 2021.
-
Disentangling Syntax and Semantics in the Brain with Deep Networks
Authors:
Charlotte Caucheteux,
Alexandre Gramfort,
Jean-Remi King
Abstract:
The activations of language transformers like GPT-2 have been shown to linearly map onto brain activity during speech comprehension. However, the nature of these activations remains largely unknown and presumably conflate distinct linguistic classes. Here, we propose a taxonomy to factorize the high-dimensional activations of language models into four combinatorial classes: lexical, compositional,…
▽ More
The activations of language transformers like GPT-2 have been shown to linearly map onto brain activity during speech comprehension. However, the nature of these activations remains largely unknown and presumably conflate distinct linguistic classes. Here, we propose a taxonomy to factorize the high-dimensional activations of language models into four combinatorial classes: lexical, compositional, syntactic, and semantic representations. We then introduce a statistical method to decompose, through the lens of GPT-2's activations, the brain activity of 345 subjects recorded with functional magnetic resonance imaging (fMRI) during the listening of ~4.6 hours of narrated text. The results highlight two findings. First, compositional representations recruit a more widespread cortical network than lexical ones, and encompass the bilateral temporal, parietal and prefrontal cortices. Second, contrary to previous claims, syntax and semantics are not associated with separated modules, but, instead, appear to share a common and distributed neural substrate. Overall, this study introduces a versatile framework to isolate, in the brain activity, the distributed representations of linguistic constructs.
△ Less
Submitted 15 June, 2021; v1 submitted 2 March, 2021;
originally announced March 2021.
