Scientist Stories: Timothy Springer, Protein Engineering and Machine Learning

Liz and Debbie, thank you so much for the invitation to be here today virtually. Um, and I'd like to talk about a personal perspective on mental representations, deliberate practice, and what machine learning can do for biology and medicine. Although I've had a long and fulfilling career, what I tell my friends these days is that I feel like I'm just coming into my prime now. Knowledge in my field is accretionary, and I'm learning so much from colleagues in science and business that it imbues everything I do with excitement. This kind of learning is what I want to focus on in this talk. How can we as individuals and collectively as scientists continue to develop our expertise throughout our lives? The answer to this question was really brought home to me by this book, the most enjoyable and instructive one I've read in a long time. Erickson was a psychologist who studied the mental representations that enable development of expertise in some specialized fields. Expertise even correlates with changes in the size or wiring of specific structures in our brains. In areas with over 100 years of experience in what the best training methods are, like music and chess, expertise strongly correlates with the number of hours of deliberate practice. But expertise is not correlated with IQ, and chess grandmasters have only average IQs. Franklin illustrates what is meant by deliberate practice. He developed his writing skills by reading essays in The Spectator and then rewriting them from memory. This revealed the areas in which he needed to improve and was a form of deliberate practice. However, in chess, he only practiced playing with others at the same level and never improved. In chess, expert teachers and studying the games of grandmasters are the most effective deliberate practice methods, and Franklin had neither. In order to develop expertise, one needs to be challenged and moved out of one's comfort zone. Teachers can do this. Grandmasters have studied the chess piece arrangements from real games. They see patterns such as lines of force. These patterns are examples of mental representations. Mental representations store large chunks of information in efficient small pieces. When challenged to recreate the positions of pieces on a chessboard, grandmasters excel with positions from an actual game, but do no better with randomly arranged pieces than a novice. These mental representations of theirs may be examples of the powerful advantages that machine learning can bring to biology. Now, scientists have only IQs slightly higher than average, and IQ does not correlate with their productivity or impact on a field. In fact, many Nobel laureates would not have qualified for Mensa. This emphasizes the importance of deliberate practice. As Anders Ericsson states, Practice is the single most important factor in determining a person's ultimate achievement in a given domain. And the importance of the deliberate aspect of practice and the quantitative relationship between hours of practice for human expertise gives me great hope for the future of machine learning. Now, the fourth chapter of Peak, The Gold Standard, describes how highly developed fields such as classical music and mathematics have broadly accepted training methods in which teachers are of central importance. After I finished this chapter, I panicked. It has been decades since I've had teachers and mentors. Luckily, I was trained well at Berkeley and had three outstanding mentors, 2 of whom won Nobel Prizes. But at this point in my career, which I'm at right now, how was I going to get better? I then realized that I had been using deliberate practice all along. And I want to share one instance with you, which occurred during my transition to becoming a structural biologist. Now, this is a great book and it really should be entitled Introduction to the Important Mental Representations in Protein Structure. I spent about a year with this book and studying protein folds with stereomolecular graphics. Earlier in my career, I had discovered proteins called integrins. Integrins have 7 sequence repeats of 60 amino acids with few disulfides. All extracellular proteins with domains this size or smaller require multiple disulfide bonds for stability. These mental representations suggested to me that the 7 integrin repeats did not fold into 7 different domains, but must come together to form a single domain. But what kind of domain could that be? And after months and months of work, I could not figure it out. And then I reread this book, and I found just what I was looking for in this illustration that comes directly from the book. But there was a problem. This is a beta propeller domain, but it has 6 repeats and not 7. The problem was, could beta-propeller domains contain 7 rather than 6 beta sheets? Well, I got the answer to that very quickly. Indeed they could, and several structures of those had already been solved, and they could serve as templates for me to build a model of this domain in integrins. But I didn't know how to build a model, and learning this was extremely important to my later career in structural biology. Because it gave me a perspective that's orthogonal to the thinking of most structural biologists. And in another example of deliberate practice, I recruited teachers, even people I had never met, who I fortunately found were willing to share their expertise with me. The next problem was, how could I align the sequence of the integrin to the sequence of the beta-propellidomain when there was no detectable sequence homology? I did manage to figure this out and was helped by rare examples where within a beta sheet there were two cysteines that I predicted would be disulfide bonded to one another. This helped because it meant that the sequences had to be aligned such that the two cysteines in the sequence would have to be close enough together in the three-dimensional structure to covalently bond to one another. This is an early example of evolutionary coupling for which Debbie Marks has found a powerful mathematical model. She and her colleague Andrew Kruse have created highly accurate structures by incorporating this model into machine learning. Such coupling is now widely used by machine learning in protein structure prediction, as exemplified recently by the success of DeepMind in CASP-14. But we need to exercise extreme caution with mental representations because they can be highly biased and wrong, as shown by Daniel Kahneman in this book, who won the Nobel Prize for showing that judgment and decision-making are often not rational because they are fast, intuitive, and emotional. And use what Kahneman calls fast thinking. Now, in science, hypotheses are important mental representations, and unfortunately, investigators show strong bias in selecting experimental findings that agree with their hypotheses. They're really not supposed to do that at all. They're supposed to reject their hypotheses if they're shown to be wrong by experiment. This intuitive fast thinking by too many scientists probably explains why about 70 to 90% of findings in biologically relevant medical research that pharmaceutical companies try to reproduce can't be reproduced. And I should point out that these mental representations that develop through fast thinking need to be complemented often by us humans, and I'm sure by computers, in slow thinking where we really go in and study a problem in detail. So to be successful in science in the long run, and also in starting companies that improve human health, objectivity and testing every idea by at least 3 independent methods is essential. As Einstein said, "Subtle is the Lord." He was talking about physics, but biology is even more so. Biology rarely turns out to be as simple as we expect. If we are to be successful, we must be flexible enough to revise our hypotheses many times in developing understanding of a deep biological process. Now, I have time just to describe two of many possible examples of applications of machine learning to solving important biological problems and discovering new drugs to help patients. Antibodies are one of the most important tools in all of biology and also some of the most important medicines for treating cancer and many other human maladies. I have co-founded and funded the Institute for Protein Innovation. Which is making a new kind of tailored antibody. It's surprisingly difficult to computationally design the loops in antibodies that come together to form a binding site for foreign antigen. Machine learning and progress in structural understanding of antibodies has now reached the stage where specific antibodies could be designed. In fact, we want to recruit people who do machine learning to the Institute for Protein Innovation. There are many high-value applications that include replacing opiates with antibodies to a receptor that's important in pain neurons. Bacteria and other pathogens have evolved many new proteins to evade host defenses. Some of these would be highly useful for treating autoimmune diseases, and this has now come to be possible. However, not all therapeutically useful proteins are useful to bacteria. So in other words, many types of useful therapeutics are not found in nature. Wouldn't it be exciting to develop them with the help of machine learning? I think we can, but we need to go beyond what DeepMind has done, determining what a specific sequence folds into. We need to actually first know what kind of a fold and function we want, And then we need to find the sequence that folds into that. That, that's a different problem that will take more time, but I'm sure we'll get there. Now, I frequently get asked by investors about the future of biotech. I answer that it's very bright because I believe that so many new advances have only become available recently or even just beginning to be made. For example, immuno-oncology and mRNA therapeutics and vaccines. In fact, I believe that if we call the 20th century the century of physics and the atomic bomb, the 21st century is the century of biology. In the current pandemic, we are seeing how nations are arming themselves, not with weapons of mass destruction, but with vaccines. One of the greatest virtues of science is the interchange of ideas and personnel on an international scale. By working together, we can be so much better. I hope that science can contribute to bringing the world together in a common fight against diseases and to meet other scourges like global warming. And I do strongly believe that machine learning has much to contribute to this 21st century of biology. Thank you.