AI Mind Reading Experiment!

Dear fellow scholars, this is two minute papers with Dr. Zsolnai. I can't believe I am saying this, but today we are going to read human minds by using AI. Yes, that's right. It really happened. So how do we read brains? Well, here we are mainly interested in a peaceful, non invasive method. And that is going to be an functional magnetic resonance imaging. This is where you go into a tube, are asked to stay still and your brain activity will be red. We get a result that looks something like this. So what is this good for? Well, these are cross sectional views of the brain slices, if you will. And not only that, but blood flow within the brain can also be highlighted. This gives us an idea about what parts of the brain are activated and when. So for instance, we can show someone an image of a familiar face and an unfamiliar one and see if there is a difference in the brain activity and what that difference is. That sounds great. However, not so fast. With these peaceful, non invasive techniques, the readings are not as precise and there is so much more noise. And now hold on to your papers because this paper did something that I did not think humanity would ever be able to do. So here is the mind reading experiment they propose. Chuck some folks into an FMRI machine, show them images and then make a brain reading. Now this brain reading would have to be converted to an image, not just this noisy blood flow information, which sounds almost impossible and it is almost impossible. The previous work from just five years ago did something like that. These images went in and this came out. This is a reconstruction of what scientists think the brain saw in terms of shapes. There is perhaps some correlation between the two, perhaps they are related, but I feel like I am grasping for straws here. And then four years later, two really promising works appeared that could reconstruct these images from the brain. An incredible scientific achievement. However, still very blurry images. Get this in this new work. At the risk of simplifying a little, the idea is to take these brain readings and plug them into the legendary text to image AI system stable diffusion. So what do we get as a result? And now is the point where I fell off the chair when reading this paper. Here are the new results. Wow, now we're talking. These are so much better. The readings show a strong correlation of the real images that have been shown to the subjects. And wait, it gets better. It also works for video. That means that we can show these subjects all kinds of videos. And after the brain reading reconstruction we don't get back exactly what was shown, but the results are so much higher. Quality than before. Now, evaluating these results is a tough matter. There are fundamental questions. For instance, for a perfect reconstruction, are these two supposed to be the same? Good question. Also, the FMRI machine and the reconstruction algorithm both have their limits, and the list goes on. A healthy dose of skepticism is the hallmark of the wise scholar. However, for now, we can compare the reconstructed results to the ground truth through one visual inspection or two through mathematical ways of comparing images to each other. With both, this new technique has been shown to be so much better than the previous papers. And as every good paper does, this one also raises important new questions too. For instance, does it work for a diverse set of topics? It does. And what I really want to know how similar would these images be for different people? What do you think? Well, I am delighted to tell you that there was an experiment for that too. Absolutely lovely. It almost feels like we are living in a science fiction world. My brain is also bubbling with ideas. I wonder what that would look like in a video form. So is this work perfect? The answer is no, not perfect. Not even close. There are still lots of limitations and many failure cases and we should all exercise a little critical thinking for works like this. But once again, the help of AI research makes the almost impossible seem possible. What a time to be alive. Weights and Biases provides tools to track your experiments in your deep learning projects and they now have a one hour course with the legendary scientist Andrew Ng on how to perform things that we are talking about in these videos. For instance, they teach you how to train diffusion models, how to run and evaluate large language models, and even fine tune them for your tasks and more. You can even run these code snippets as you are watching the series. It is fantastic. Make sure to visit them through WNB Me papercourse or just click the link in the video description and you can take this course for free. Our thanks to Weights and Biases for their long standing support and for helping us make better videos for you. Thanks for watching and for your generous support and I'll see you next time.