YouTube science-communication monologue explaining a paper that reconstructs audio (Pink Floyd’s “Another Brick in the Wall”) from intracranial EEG recordings of 29 patients using neural-network decoding. The presenter describes experiment design (song → auditory spectrogram → neural recordings → neural-network-based spectrogram reconstruction), relates reconstruction quality to electrode count/noise tradeoffs, and connects this work to related brain-to-text (imagined handwriting) research. The video ends with a sponsor ad for Lambda GPU Cloud.
Questions this video answers
29 patients.
"Another Brick in the Wall" (Pink Floyd).
Intracranial EEG (electrodes placed on the brain surface; invasive).
A neural-network-based decoder (digital neural network).
Reconstruction quality generally increases with the number of electrodes but also increases noise; decoding method affects outcome (chart described).
Chapters
Today you are in for a treat because we are going to perform Brain two audio. Yes. Listen to music straight from your brain? Kind of. And more. Yes, I hear you asking, doctor, how is this even possible? Well, this new paper showcases an EEG based technique to perform this science fiction like experiment. Normally, EEG electroencephalography means that we place electrodes on the scalp and then we listen to the brain from the outside. And by that I mean looking at the electric activity. However, here this work uses intracranial EEG that requires electrodes to be on the surface of the brain, thus making it an invasive technique that makes scientists able to listen to the brain straight from its surface. So what can we do with that? And the answer is absolutely miraculous things. Let's look at the experiment design and see what happened. First, a song is played to 29 patients. This was another brick in the wall, the classic from Pink Floyd. Then an auditory spectrogram is created from this song. This enables us to look at the song and understand a little better where we are and what is happening. Then scientists recorded the neural activity of the brain at the points where the electrodes were place and created a spectrogram from information that was obtained from the brain. This is a decoding step and the one in this particular diagram was done by a neural network decoding real neural information by using a digital neural network. What a time to be alive. Now hold on to your papers because we are going to look at the two spectrograms and. Oh my goodness. The information that was read from the brain indeed refers to the same song. No question about it. Due to copyright constraints, I can't play it here, but you can listen to it through the link in the video description. And I have to say that of course it is not hi fi quality, but if you know the song, you can tell what is being played in the brain and even sing the lyrics to it. Absolutely incredible. The paper also showcases a ton of little golden nuggets of information. I love this chart that tells us exactly how the quality of the reconstruction increases with the number of electrodes, little measurement points being used on the brain. Adding more of these typically increases the resolution of the information we get, but also increases the noise as well. So here, a proper way of decoding helps a great deal. This paper is an interesting showcase of what is possible through the power of research. And since this data has to be read from the surface of your brain, I would not worry about anyone eavesdropping on which scholarly paper you are thinking about. For instance, a couple Hundred episodes ago, we talked about an absolutely amazing paper that allowed brain to text transcription. It took patients who had a paralyzed hand. And get this, all the test subjects have to do is imagine writing the letters and they magically appeared on the screen. And now hold on to your papers, fellow scholars, and just look at how quickly it goes. 90 characters per minute with over 94% of accuracy, which can be improved to over 99% of accuracy with with an additional autocorrect technique. That is absolutely amazing. We can learn a ton from these papers. For instance, in that one we learned that it is quite remarkable that even years after paralysis, the motor cortex is still strong enough to be able to be read by a brain computer interface well enough for such typing speed and accuracy. It truly feels like we are living in a science fiction world. Imagine how many people around the world we could help with this. So today, brain to text is possible and brain to audio is also possible. But here comes the best part for people who are unable to communicate with us through speech. Now they can not only write to us, but with this new paper we can even reconstruct the musicality of their speech as well. Straight from the brain. Imagine if we could have heard the late Stephen Hawking talk like that. How amazing would that have been? Wow. If you're looking for inexpensive cloud GPUs for AI, Lambda now offers the best prices in the world for GPU cloud compute. No commitments or negotiation required. Just sign up and launch an instance. And hold onto your papers. Because with the Lambda GPU cloud, you can now get on demand H100 instances for just $199 per hour. Yes, $199. And they are one of the first cloud providers to offer publicly available on demand H100 access. Did I mention they also offer persistent storage? So join researchers at organizations like Apple, MIT and Caltech in using Lambda Cloud instances, workstations or servers. Make sure to go to lambdalabs.compapers to sign up for one of their amazing GPU instances today. Thanks for watching and for your generous support and I'll see you next time.
