AI for Science

Welcome back to DeepMind: The Podcast with me, Hannah Fry. For this series, I've been speaking to researchers at the cutting edge of AI to find out what they're working on and the implications it could have for all of us. In the last 4 episodes, we've been looking ahead to the long-term future of AI, including some of the ideas that researchers hope will bring them to artificial general intelligence. But in the next two episodes, we want to share with you some of the ways that AI is already being put to work along the way. Here's DeepMind CEO Demis Hassabis.

My personal reason for working on AI was to use AI as the ultimate tool to accelerate scientific discovery in almost any field. And I think AlphaFold is our first massive example of that.

If you heard episode 1, you'll know all about AlphaFold, DeepMind's state-of-the-art system that can accurately predict the 3-dimensional structures of proteins. But AlphaFold isn't the only science project within these walls. Step inside the labs and you'll find researchers scrutinizing DNA to understand the mysteries of life. Hunting for new ways to harness nuclear energy, or putting AI to the test in some of the most mind-expanding areas of mathematics. So, what are you waiting for? Come on in. This is Episode 6: AI for Science. Pushmeet Kohli, who we heard from in Episode 1, oversees DeepMind's AI for Science programme across the natural sciences. And when it comes to a list of areas that are being worked on, he is exactly the man to ask.

Biology, proteomics and genomics, in quantum chemistry, like material design, fusion, fundamental mathematics, ecology, weather.

Now that is a fairly intimidating list, but over the course of the next 30 minutes or so, I'll be walking you through as many of them as we can get to, to give you a sense of the potential for AI to make a difference to science. First though, you might be wondering why a company that made its name getting AI to play computer games became involved in these serious scientific subjects. And that is where I started the conversation with Pushmeet.

Science has been in the DNA of DeepMind from the very start. It's more of the case that as we sort of built up these systems and proved them on games, we then started thinking about now is a good time to actually stress test them on the real scientific challenges that society is facing.

For Sarah-Jane Dunn, a research scientist on the science team, DeepMind Science Program is fundamental to the company's aim of solving intelligence to advance science and benefit humanity.

That's quite an abstract concept to some people. The point is you have techniques that can tackle some of the hardest problems that have eluded the brightest minds for decades. And so I think being able to show people the power of these technologies to really help humanity, that for me is one of the most valuable aspects of what we're doing in science.

What the AI for Science team does is try to identify suitable scientific problems for AI to work on that will go on to have the biggest impact.

One way of thinking about it is this idea of the tree of knowledge. There are some problems which are the so-called root node problems, which unlock so many other problems downstream. Protein folding was one such example. Once you understand the structure of proteins, that has implications for developing new drugs or coming up with new enzymes for breaking down plastics. Another example would be if you had a great source of energy, other problems like access to clean water become much more tractable once you have unlimited energy at your disposal. And then once we have sort of isolated some of these root node problems, we then think about what is the role machine learning and AI has to play.

And now that DeepMind has made significant progress on protein folding, it's turning to some of those other root node problems that Pushmi mentioned. One of them is nuclear fusion. It's hard to emphasize just how big an impact a breakthrough in nuclear fusion might have. It is the long-held dream of scientists around the world that one day we would be able to fuse two hydrogen atoms together and in doing so create a totally unlimited, totally clean clean and safe supply of energy. It would spell the end of our climate crisis. But before we get too carried away, I should tell you that nuclear fusion is really hard. To get atoms to fuse, you first have to heat them up until they form something called a plasma, a state of matter that is so hot that the electrons are stripped from the atoms. Here's Demis Hassabis to explain.

One of the really hard things about fusion is how do you contain this plasma that's like as hot as the sun, and it would burn anything obviously that it touched. So the way you have to contain it is in a magnetic field. The problem is that the plasma is almost in a chaotic regime, so at any moment a bit of it might just sort of shoot out in a certain direction, and you have to change the magnetic field quick enough to respond to that, to keep hold of the plasma for multiple seconds.

The plasma is held inside something called a tokamak. It's like a giant metal donut big enough to walk through. The magnets around the edge create a field strong enough to suspend the plasma in the middle away from the sides. But the plasma is wobbly and unstable., and as soon as it touches anything, it's game over. The comparatively cold sides of the tokamak saps the energy from the plasma, and the heat dissipates almost instantly.

Up till now, what people have done in the physics world is they've handwritten mathematical controllers for what the plasma might do, whereas our system, what it's learned to do, is predict what the plasma might do. From the shape of it, and then almost ahead of time change the magnetic field to react to what it thinks is going to happen. And even more excitingly, we're actually doing like plasma sculpting, so we can actually split it into two or elongate it. It's almost like ice sculpture but with plasma.

This is hot as the sun.

Yeah, so hot as the sun.

But as Pushmeet explains, it's not just about keeping the plasma stable.

It's also about which configuration of the plasma you are keeping stable. There are certain configurations which produce more heat, and what reinforcement learning now allows you to do is basically you can say, I want this funky configuration because I think that it is going to be more stable, or it will produce more heat capacity, and so on. And the AI is like, yeah, I can do it for you, rather than sort of a human spending a year of their time sort of thinking about how to control the different current in the coils and how to maintain the right voltages and so on for that particular configuration.

I see. So it's a shortcut, basically.

It accelerates the whole research cycle.

Amazing. And as if protein folding and nuclear fusion weren't enough to keep the AI for Science team busy, they've been pursuing a raft of other projects, including in the field of ecology. This has taken AI out of the science lab and into a setting where you might not expect to find it at work.

Where we're working in the Serengeti is very classic Lion King East African landscape. So you have the rolling grasslands, it's dotted with those iconic acacia trees, broad open savannas.

Meredith Palmer is a conservation biologist at Princeton University in New Jersey, but you'll usually find her with a pair of binoculars out in the field.

You have something like 700 species of birds, 50 species of large mammals that are traversing around the greater Serengeti ecosystem. So down from Tanzania, up into Kenya, in this kind of endless circle chasing the rains. And when the migration comes back into town, you can hear the bleating of the wildebeest, the sounds of the flies that are following the migration around.

This annual gathering of animals, the Great Migration as it's known, is one of the last in the world and one of many reasons that ecologists are so interested in the Serengeti.

We have lions, but we also have cheetah and wild dog and hyena and leopard, and all of these species are trying to coexist And so to understand all of the interconnections between these wildlife, we need to be studying these wildlife communities.

Unfortunately though, climate and agricultural change, as well as illegal poaching, are all having a growing impact on the local ecosystem. About a decade ago, to help them monitor changes in the Serengeti, ecologists installed devices known as camera traps. Across a 1,200 square kilometer area of Serengeti National Park.

Camera traps are small remote cameras. We can strap them to a tree and leave them running 24 hours a day, 7 days a week for months or even years. The camera traps are triggered by the heat and motion of passing animals to take pictures. We can examine the images to see what animals are in them and what these animals are doing.

As you might expect, these camera traps take a lot of pictures. Lots of fantastic candid shots, and lots that show the back of an animal or some shape that's a bit hard to discern. Imagine letting a 4-year-old run loose with your phone and then scrolling through the camera roll at the end. It's a bit like that.

These camera traps are producing 10,000 to 20,000 images a month. So we as ecologists are really facing this massive deluge of data. And the big issue for us is that we can't use the images straight out of the camera trap to address our research and conservation questions. Someone has to go and look at each and every one of these tens of hundreds of thousands of images and write down, you know, this image contains 3 adult zebra and 1 baby zebra. I once calculated that it would take me 7 or 8 years to process a single year's worth of image data.

Faced with all this data, Meredith and her colleagues became early adopters of citizen science.

We essentially crowdsourced the classification process, and you could look at our camera trap pictures and identify all of the animals and behaviors that you saw in our field data.

After a while, though, a problem arose. As the number of citizen science projects exploded, the Snapshot Serengeti project, as it was called, could no longer attract enough volunteers to manually go through all of the images that needed labeling. Snapshot Serengeti's impressive dataset of labeled wildlife images caught the attention of researchers at DeepMind. They got in touch, offering to train a computer vision algorithm that would automatically classify the thousands of photos taken by the camera traps, identifying which particular species of gazelle, for example, appears in a photo.

It blows my mind how incredible AI can be for solving these kinds of problems. I've had moments where I've looked at an image and had a computer identify a species I, you know, on first glance didn't even recognize was there. So there's some animals that the AI is 99% on all of the time. However, if we don't have a lot of labeled images, the algorithm doesn't do so well. So there's a lot of these cryptic, rare little species— aardvarks, surilas, servals, genets— And when we only have a couple hundred images of those, the computer can get a little wonky just because it doesn't know what to look for.

Computer vision systems, and pretty much all AI systems come to that, are only as good as the data they're trained on, which also explains why the AI sometimes struggles with those creatures at the top of the animal kingdom.

One issue that we discovered using human volunteers to classify our data is that people really, really, really want to see lions. We've had images where it's a warthog sunning itself, and our citizen scientists have very confidently classified that image as containing one very impressive male lion. And no, I'm sorry, it's not. It's a warthog. We do have a lot of checks and balances for making sure that we're feeding We're feeding the AI the very best labeled image data that we can, but some of these things slip through the cracks.

The community of Serengeti ecologists are already starting to see the benefits in their monitoring efforts. Tanzania used to have an enormous population of black rhinoceros, but that has dwindled to fewer than 100. In 2019, 9 black rhinoceros were reintroduced into the Grometi Game Reserve, a part of the Serengeti ecosystem that is monitored by camera traps in conjunction with AI technology.

We'll be able to look at if migration routes change. We'll also be able to document any impacts of new mega herbivore effects on the surrounding human communities. We've documented several of the rhinoceros consorting with each other. So we anticipate that we're probably going to have a couple of baby rhinoceros to look forward to in the next year or two.

For Meredith, the future is not for AI to replace ecologists, but to complement them.

The way we've been doing ecology to date, you know, out there in the field with our binoculars and our notepads, watching, you know, a single warthog all day, like, it's not fast enough.. And the more that we can get a helping hand from AI, the faster we can identify the problems that are driving change in the ecosystem, the more targeted conservation interventions that we can make to save and protect and rebuild these ecosystems.

We've heard how computer vision systems are being used to make sense of biology and wildlife on a vast scale. But as we heard with AlphaFold, DeepMind is also using AI to understand life at a microscopic scale. Back in the science lab, researchers have been trying their hand at genomics, an area of biology dedicated to understanding human genes and DNA. DeepMind's genomics work is still in its very early stages. So what are Pushmeet Kohli's hopes for the future of this research?

The implications could be very wide-ranging. If you truly understand biology, then you understand how all cells behave, even cancer cells. So you might have better treatment for cancer. You might have better ways of creating new organs for transplantation.

What you're describing here really, though, is a potential step change in all of healthcare and medicine.

That is the implication of truly understanding biology. We have in some sense been observing biology, but to truly leverage biology, that's something that humanity is only starting to understand the implications of.

So Are you ready to get up to speed on a brand new bit of science? I'm game if you are.

I started out life as a mathematician, but when I was thinking about what to do, I wasn't so taken with the idea maybe of studying fluid flow down a pipe for 3 years.

No offence to— To me, which is exactly what I did. Sorry. This is the delightful Sarah-Jane Dunn, a research scientist working on the genomics project.

Genomics, it's a domain of biology, and it's most interested in understanding the connection between your genotype— that's basically your DNA sequence— and phenotype. And that's everything from how tall you are to what color hair you have to maybe whether you have some particular disease or not.

Genotype is what you see in the DNA. Phenotype is what you get. I have the MC1R genotype RS185005, which gives me the phenotype of red hair. Genotype CFTR delta F508 gives the phenotype of cystic fibrosis. Those, however, are the examples where the connection between genotype and phenotype is clear and established. Unfortunately, such examples are the exception rather than the rule. Take, for example, the BRCA1 gene, which is linked to the development of breast cancer. Not all people with the BRCA1 mutation will go on to develop breast cancer, but there's currently no way of knowing which people with the gene are more at risk. In some cases, this has led some women with the BRCA1 gene to have surgery as a precaution. The hope is that a deeper understanding of genomics may provide important additional context about what puts someone at increased risk or not.

If we could unpack more about how the rest of your DNA sequence can tell us about the progression of that disease, then we might be able to make more nuanced diagnoses about these kind of things. And then projecting into the future, How might you be able to change that before the disease actually develops? How might you be able to develop the most effective, quick treatments? Those are all questions that are open to us after that.

Research in genomics builds upon an important foundation known as the Human Genome Project. Starting in 1990, the project took the full set of DNA of one anonymous human their genome, and over the course of a decade went through every molecule one by one. Every single one of the 3 billion chemical base pairs, A, T, C, and G, that combine together to give some 20,000 or so genes, and a whole lot more besides, thus providing the entire manual for how to make a human. But in some ways, this was just the starting point.

The Human Genome Project successfully was able to compile this whole sort of cookbook for a human. You have bought this book, and there's a saying, but you don't know how to read it. That's what scientists are sort of struggling with.

You can think of the amazing human cookbook as a literal book, by the way. One of the early versions of the Human Genome Project was printed and bound as part of an art project. And it stacked up as some 20 phone book-sized volumes, where every page inside was a blur of As, Ts, Cs, and Gs. And therein lies part of the problem. We still have very little idea about how much of it comes together to form the cells, tissues, and organs in the human body. It is a list of ingredients, but it's not in order.

And that's what we are trying to understand. How do you decipher the actual sort of cooking process?

So you've got one end, some flour, eggs, milk, and a cow. And at the other end, you've got a beef pie, and you're like, what the hell happened in between? Yeah, exactly. In humans, only about 2% of our DNA actually goes towards our genes. The remaining 98% of those 20-odd phone books' worth of biological instructions are what they officially call the non-coding region.

Less officially, biologists have referred to it as junk DNA. You know, that shows how much we were interested in it. But after the Human Genome Project was completed, it became obvious that this so-called junk DNA was actually really important.

By now we know what some of the junk does. Some of it is structural stuff, like the scaffolding needed for when a cell is working. Some of it is just random nonsense. But a lot of it is on and off switches, a series of regulators that tell genes when to get to work.

What's interesting is that, you know, all of the cells in your body share that genome, but they make use of it in different ways. And they do that by expressing some genes and not others.

Every cell contains a complete copy of every gene, but will only require a handful of genes to be active at any one time. A gene that prompts cell division is no use in a tissue which has finished growing, otherwise it could lead to a tumor. A gene that is active in building hair or teeth has no role in the cells that make up the liver or the heart. The hope in all this is that if you can understand the on and off switches, as well as everything we already know about genes and proteins, it can help biologists to better understand the mechanism by which certain diseases take hold in the body. Take, for example, sickle cell disease, an inherited health condition affecting red blood cells. There are, of course, certain signs of this disease in someone's phenotype. If you look at the red blood cells under a microscope, those that belong to someone with the disease will take on a characteristic shape, like a crescent moon. Thanks to the Human Genome Project, we now also know that there are certain changes in the DNA that are common among people with sickle cell anaemia. What's still unknown, and where genomics can help, is how that sequence of letters in someone's non-coding DNA translates into how the disease will progress in individual cells.

If you're wanting to then understand how this disease actually arises in the body, and then ultimately how you might treat it, you need to go through these layers of information. You need to understand, well, what does that change the DNA sequence actually do within the cell? Does it mean that the cell can no longer express certain genes? Does this give us targets for what we might be able to attack in that cell to develop kind of drug treatments. So it's about understanding from that very first clue about what might give someone a disease, how it kind of bubbles up from that basic level of DNA.

That is where AI and DeepMind's work on genomics fits in. There is some precedent here already. Scientists have a whole series of special cases where they've already cracked the connection between DNA sequence sequences and cell function. These studies all serve as a solid foundation of biological data for DeepMind's foray into genomics research.

We're trying to build AI that essentially can understand the genome. The fact is we now have the kind of level of data where deep learning can really take hold, and we've been able to build together the kind of deep learning models that are able to read a DNA sequence a bit like you might read a piece of text.

As a first step in its genomics work, DeepMind is using a relatively new idea in the world of machine learning, a type of architecture called a transformer. It's useful to think of it as a sort of translator. It reads in a string of letters from DNA, and based on the sequence of As and Ts and Cs and Gs, it will translate it into a prediction for how the genes will actually manifest themselves. To do language translation well, you need to pay most attention to the keywords in a sentence, and the transformer uses the same idea applied to DNA. As it runs through the code, it holds on to the most important parts of the sequence, the bits that are most likely to provide the relevant bits of context. And it pays less attention to all the filler bits, the DNA equivalent of all of the 'ands' and the 'ins' and the 'the's' in a sentence.

You need more context to be able to make better and better predictions because some of the things that can influence what's happening are positioned further and further along the DNA sequence. And so the transformer architecture that we built is able to swallow in about 10 times as much of that DNA sequence.

That's 10 times more of the DNA sequence than previous AI models could read.

By being able to extend how much context the model had about the DNA sequence, we were able to increase the accuracy of our predictions about gene expression from sequence.

This all sounds pretty positive. But before we make it sound like machine learning or AI is some sort of silver bullet for all problems in biology and science, it's important to sound a note of caution too. Unfortunately, there isn't a big button labelled 'machine learning' in the corner where you can just gather any old data, hit the button, and get out great science.

To use AI effectively, you have to ask the right question. That, for me, is one of the biggest challenges, to say, okay, I know that there's a big question in the field about how we can, you know, differentiate a progenitor cell into a heart cell because we'd like to be able to make them in the lab so that we can treat people with heart disease. But how can you take something like that and then wield AI effectively? It's being able to understand, first of all, what kind of question you should be asking, and then what kind of architecture is going to be able to deal with those data. But then also, how do you know if your model's getting better or worse? So it very much is still an area of research.

The other thing about using AI, the sophisticated pattern hunter that it is, is that it doesn't always give you a good answer for why those patterns appear. Do we have to be nervous about making predictions without really understanding the causal mechanisms?

Yes, I think so. But it doesn't mean that these kind of tools aren't valuable. The thing I like about AI is that it can allow us to handle some of the complexity that biology throws at us. And often when we're dealing with biological data, we're dealing with very noisy data and data that has been perturbed by just experimental error. And so, if we have the tools that can somehow surf over some of those bumps in the data and try to really pull out those meaningful patterns, then the thing that we have to be really rigorous about is how we interpret it. And it's not the only tool that should be deployed. But if it gives us pointers in the right direction, if it steps us forward that one piece, then that is still valuable.

But it's all very well for computer scientists to turn up in some new scientific domain hoping to plant their flag. As we've heard, machine learning techniques like AlphaFold and this genomics work build heavily on existing scientific data that is often obtained using painstaking experimental methods. So how do other scientists respond when they hear that AI could be applied to their domain?

Scientists are a very competitive bunch, but they are very collaborative as well. If they are convinced that the approach that you are proposing is an approach worth pursuing, they are interested in listening to you.

Have you ever struggled to persuade people though?

So when we started looking at applying machine learning and AI to pure mathematics, we encountered a number of mathematicians who were very, very keen seen and became really good collaborators. But there were a few who, on matters of principle, thought that mathematics is a very human endeavor and a machine has no role to play in it.

I'm shocked. I'm shocked that some pure mathematicians might not welcome the future with open arms. You're probably wondering what that pure mathematics project was all about. Well, the first thing that you need to know is that in pure mathematics, proof reigns supreme. Someone might have a conjecture, like there are an infinite number of prime numbers, for instance, but that means nothing until by a chain of logical statements it can be proven to be indisputably true for all eternity.

So what we were interested in is, can a computer have that intuitive ability to propose a new conjecture. What we started looking at with some of our sort of collaborators is about different aspects of knot theory and topology which were suspected to have some interactions, but nobody had sort of discovered those relationships.

Knot theory, you will not be surprised to hear, is all about the mathematical study of knots. I know the name might not make it sound exciting, but you're just gonna have to take my word for it on this one that the whole field is a glorious playground of bending and twisting loops of imaginary string. There are different ways to describe these knots, an algebraic way and a geometric way. And it's quite hard to translate from one to the other. But then AI started hunting for connections.

And that resulted in a new conjecture which had never been seen before. And not only that, the human mathematicians then managed to prove that conjecture. So in some sense, it's basically humans who are proving the conjectures that a computer has come up with.

Did that persuade the grumpy pure mathematicians?

I don't think we went back to them.

And that is just a flavour of some of the science applications that DeepMind is currently hard at work on. From ecology to genomics, protein folding to nuclear fusion, the ways in which AI could make a genuine difference to our lives in the near future are increasingly clear. And there are so many more projects that we didn't have time to cover in this episode. So if you'd like to find out more about the AI for Science programme, do take a look at the show notes or check out deepmind.com. In the next episode, we'll continue to discover how the applications of AI could have an impact on the real world, from weather prediction to voice synthesis. I'm a mathematician, author, and podcaster who's fascinated by artificial intelligence.

It's really good. I know, it's good.

Am I that breathy? Bloody hell. You've been listening to DeepMind: The Podcast. I'm Hannah Fry, and the series producer is Dan Hardoon at Whistledown Productions. We'll be back soon.