Proteomics and Deep Learning with Melih Yilmaz

Welcome to the Axial Podcast. Axial is an early-stage investment firm based in San Francisco. We partner with great founders and inventors investing in early-stage life science companies, often when they are no more than an idea. Axial is fanatical about helping the rare inventor who is compelled to build their own enduring business. Well, Meili, thanks for doing this. Um, I appreciate taking the time and dealing with rescheduling. And yeah, really excited to talk to you and record it. You're one of the up-and-coming people in proteomics and like comp bio, and I think you have a really cool story, so I want to tell it or talk about it. Maybe to start off, do a brief introduction on yourself.

Thanks, thanks, Josh. Thanks for having me. So yeah, my name is Milij. I am now a third-year computer science PhD student at the University of Washington. Broadly, I'm working in computational biology and I'm developing machine learning tools, deep learning tools to basically answer interesting questions in the context of proteomics and beyond. I honestly didn't start out as a biologist or a computational biologist. In the beginning, it was only just engineering that I was sort of like focusing my gaze on as a high school student going to undergrad. And then I did my undergrad in electrical engineering at Koç University in Istanbul, Turkey. So during my undergrad, at some point, I guess that was 2016, 2017, where sort of like the deep learning revolution was gaining pace by the day. I was really sort of like drawn into some of the advancements there. I think that the singular point that I can remember now is like overnight how Google Translate become like so much better. With recent advances in NLP. I guess, yeah, with that I was, okay, that's a good subfield that I can focus on. Machine learning and data science was broadly something that I was really interested in. I did a bunch of online courses. I did some self-studied projects before pursuing any coursework because usually those are more advanced courses that juniors or even senior students take. So it was a bit of a self-start. I dabbled there a little bit, did like a few internships in the industry. And around the same time, I— so basically, in my university, there weren't a lot of different opportunities to get involved in research as an undergrad, but I was aware of like research as a potential way forward, as like a potential career track. And I basically got involved with one of the labs in electrical engineering that Mohamed Pekal. So that initial experience was around image and video processing, again using like deep learning tools. So that's how I got started. But over time, like after my first year in that lab, I was sort of like really excited about research overall. Like I was excited about the concepts, about the idea, and like using machine learning tools and building machine learning tools. But at the end, I discovered I want to do something like really something applied that would basically involved me sort of like developing expertise in some other domain outside of machine learning and sort of like really bringing some of these new and cool methods and these powerful methods to another domain to solve interesting problems. And again, probably with a sort of like strike of luck, I was mainly, I was like for the summer. So this is again, the summer before I graduated, I was looking for internships. For research internships specifically. And it was sort of like really people back in my university, people were always sort of like going to the US. So I was also, I also had my eyes in the schools in the US and I landed a research internship at Stanford. And lo and behold, that was sort of like mainly focusing on applications in biomedicine. So I worked with Tina Hernandez-Bussard at their Center for Biomedical Informatics. And really that sort of like how I started to focus more on applications in biomedicine. So not necessarily biology at that point, but more broadly like human health and sort of like applications of machine learning tools and data science in that domain. And really that summer taught me a lot, not only in terms of like, oh, I want to like work on this particular problem, but more broadly, oh, like these applications are great. And also at Stanford, I had the opportunity to really talk to a lot of different people, around that domain, like around biomedicine. So come next year, when I was applying for grad school, I really had the sense that, okay, I want to work on computational biomedicine, whether that's biology or more health-oriented or clinical applications of machine learning. And that's how I basically sort of ended up in this corner of the country at UW and sort of working in that space.

Cool. Was that Stanford Slack or was it somewhere else?

It's the BMIR, the biomedical informatics.

Yeah, well, I thought you were— I remember you were— I'm just trying to remember if you worked at Slack or something. I think you've had, uh, before UW, you've had like a varied career already. What's one thing we talk about, like, get all these experiences before you went to grad school, I think. But first and foremost, what makes you pretty unique is like world-class engineer, you know, building software. Why do biology? Was it just because you had that experience at Stanford, or was there something planted before that, like, maybe attracted you?

Yeah, I mean, Stanford was pivotal, but even before then, as, as like, even as a freshman, I was interested in neuroscience, like, very broadly, from like a very amateur sort of like popular science level. So I had like those grains of interest, but I didn't like do any coursework in biology. I didn't have like any background beyond that. But really after I did this sort of like a stint at Stanford, when I came back in my final year, I started to dive a little deeper into like even basic biology. I dabbled a little bit in immunology at that point and also like synthetic biology to have a better sense of lay of the land in that realm. And at that point occurred to me, so like on one hand you have this like really cool advances in machine learning, like that defined, basically defined the decade in science. But on the other hand, you have like a host of other sort of like inventions and really sort of like strong progress on biology in general, but like biotech and probably like epitomized by the advent of CRISPR and sort of like related technologies at that point. So I was really sort of like drawn to that and had seen great potential in biology in that sense and biomedicine at large. And of course it's, I think like everyone who's working in this field would mention this at one point or another, but potential to sort of like create real impact in human lives and sort of like the opportunity to work on really difficult problems is always a part of the draw and sort of like still is to this day.

That makes a lot of sense. Do you have any like envy against like, you know, OpenAI things or DeepMind things? I just have a friend who is like a world expert in AI, like LLMs, but he works in biotech. And so it's just like sometimes I talk to him and it's like, oh man, opportunity cost. Like he loves biotech, he loves medicine, but he's also an expert in AI. Do you ever get like envy? Like, man, I could have— we're going to have this kind of hype for like maybe, you know, text-based models or something.

Yeah.

Do you ever like, how do you deal with the optionality of being an engineer in biology versus maybe, you know, being a biologist, you're just kind of stuck in a lot of ways. You can't, you know, you don't have the skill set to do a bunch of other things. Whereas when you build, you're right, offboard engineering discipline, why do something in bio? Like, how do you, how do you deal with that optionality? Is it a passion? Is it something else? Is it like conviction or just maybe curiosity?

I guess it's a blend of conviction and sort of like to some extent stubbornness, because I set out on this path like a couple of years back at this point with such conviction that I would sort of like, okay, I would need to make a significant investment to be able to sort of like move the needle in any sort of like in any field or or any subfield of biology or biomedicine in general. And in that sense, I'm still working towards that. I mean, of course, there are a lot of things happening by the week, not even by the year, but even by the week in other domains of machine learning, especially with language models. But still, I think my focus is still sharply on that realm where I'm currently working in. But of course, I think in a bit, we can talk about how the two can merge, or like that can be a, that could be like a, um, a nice interplay between the two approaches, or like two domains, or how we can like make the best use of these new language interfaces to, to discover new biology, or like make discovering new biology much easier.

That makes a lot of sense. Okay, so you get to UW, uh, I think in the Genome Sciences Department, great program, um, like very small and something I considered going to. And so why Washington? I think you come from Turkey. Did you just not know San Francisco existed or LA? You just think, you know, was it ignorance that you went to Washington, or was it like a, like a selection process that this is the best place for me?

Yeah, I mean, no, not at all. So I basically applied to maybe like 20 schools across the US, and not in the US, like in other countries as well. Had a few acceptances, and at the end of the day, I was mainly sort of like comparing between different PIs, different programs, different types of programs even, because I ended up at a computer science program at UW. But like elsewhere, I got acceptances into like computational biology, for example. But at the end of the day, I realized a few things. So I mean, first of all, I think whenever like people are choosing PhD labs or like basically, yeah, basically whenever they're choosing PhD programs, the choice of the lab and the advisor is the most important thing, like period. And in that sense, I really had the sense that there were at least a few different PIs that I would be really happy, sort of like that I could see myself working with, like really happily and like making progress on topics that I, that I would care about, like very broadly, because at that point I didn't have sort of like proteomics as a specific field that I wanted to work on, but kind of sort of like developed along the way. And also Maybe less importantly, but still important, was the fact that here it was like a computer science program. I would still be sort of like, I would still be training in a more established part or like a more central field that's closer to my training, earlier training. And in that sense, computer science sort of like was more of a draw. And of course, I had perceived Seattle to be like a nice place to live and potentially there would be sort of like other opportunities after graduation. Those also played into my decision, and I would say at this point I'm really so happy that I've come here 2 years ago.

Okay, seems like you did a little work, but did you visit San Francisco? Did you visit LA, or was it just like you didn't do that at all?

I've seen all of them. Yeah, I've seen all of them.

Okay, what's all—

Yeah, I still hold that Seattle's like probably the best to live in.

Yeah, it rains too much though. That's the thing. I visited Seattle and I was like, this is— I came on a beautiful day. I was like, it's beautiful here. But then you look at the data and it's like it rains most of the year. I was like, oh, I can't, you know, it's sunny right now in SF. So, okay, cool. So you join UDub, you're doing computer science, you have interest in biology. How did you go about picking a prom, a mentor, or do rotations? Like, how did you end up in proteomics? And yeah, I think you're from Noble and I'm blanking, Shinduri. So two great labs. How did you kind of get there? What was your kind of like that framework to like, you know, figure out what you're working on now?

So actually, when I first joined UW, I did a like a brief rotation with some other lab on like one of the different problems. I was so at that point I was mainly working on sensing problem. So it was basically based on just like Fitbit data or similarly sort of like wearable data, trying to predict some clinical outcomes, trying to sort of like diagnose flu. It was a project like in that realm. But after that rotation, I basically got in touch with my current advisor, Bill Nopal. And with Bill, we sort of like discussed a few different project ideas. So in the lab currently, there mainly two streams of sort of like research, one like on the genomic side, the other on the sort of like proteomics side. And at that point, I mean, I had no idea about like what proteomics was up until that point, only had like a very vague, very sort of like high-level idea of the problems there. But as I sort of like go a little deeper and with sort of like Bill's encouragement, I think I saw that that would be like opportunities to bring some of the recent advances in other fields of like applied machine learning to proteomics to solve like really important problems. So in that sense, it felt like a bit more immediate, like those problems felt a bit more immediate and also relatively easier to solve for someone who's just like just entering that field without like a lot of background. So in that sense, that's how I got started with the proteomics and sort of like building tools in that space. And after like a short while, Basically from like a collaboration, my co-advisor Seowon Go sort of like joined us. Seowon is, he's not a computational biologist, he's more of a sort of foundational machine learning expert. And with Phil, I'm trying to balance the two of the sort of like both develop really cool, exciting methods, but also like addressing real problems in proteomics and mass spectrometry. Okay, cool.

I think that makes a lot of sense. I think we did a conversation with a guy named Hans Stark.

You should talk to him.

He's at MIT. He's like really, really good and he got like a paper published, got 2 papers published within 2 months.

I know, I know Hansberg. Yeah, I think he's up to a great start.

He's really good and like he has these great journal clubs actually, they're really, really useful. And you've had similar success, you were able to get like some papers put out within a year or 2. How would you credit success to that? I think for Hans he had a shortcut, he had worked in that lab before as an intern or something, and then, you know, kind of slide right into grad school. I'm always a little bit jealous of the scientists who join a lab and then immediately get success, or, you know, immediately hit on a project. What would you credit, like, your early success in the lab for? Is it like project selection? Is it having worked on this problem before? I really, people I really am envious of, kind of always have been, have been the people who've been doing their work, get a paper published after a rotation.

That's the worst.

I meet people like, you got like a first author paper, not even first author, like a middle author paper just from a rotation project. Like that would kind of make me kind of realize, hey, maybe I should be doing this. At least for you in grad school, what's been kind of the— what was the key driver initially for you to like get traction?

Yeah, I think, I mean, especially for— probably applies to like all researchers, but especially for junior researchers who are just, just starting, I think an important part of it is luck. But I mean, beyond luck, I think in the first year, in the first like half year, persistence is the key. And for me, I mean, I was, I was entering this like new application domain without any experience, although I had like strong machine learning background by that point. I had the like a machine learning practitioner for close to 4 years at that point. That said, the new sort of like some of the new modalities and some of the new architectures, like, and like, really the paradigm that we were using was not really something that I had experience with at that point. So I also had to pick that up. We can, of course, talk about the contents of the research in a bit. But I guess generally, What really made the difference was I was able to sort of like work heads down for close to a year before I was like compiling a manuscript on this sort of like new topic that I was tackling. And during that time, my advisor was really patient with me. He sort of like guided me, I think, in the right direction. And maybe another important point would be like after the first few months, I was initially working on a different project in mass spec, but it wasn't sort of like going as fast. And I had realized this other opportunity that we can potentially build something like quicker and sort of like get it out before someone else does. And I think that switch also really played nicely into sort of like me getting a first publication before like before too long joining this lab.

Yeah, I think just having the background that no one else has and bringing it to another field is always pretty productive, just kind of doing things by synthesis. Like, I have something unrelated that's useful that I can bring it— like, I'm sure if you want to into like immunology, you probably have a lot of success as well, or you went to neuro or, you know, uh, whatever. Uh, and so from proteomics, you know, you work on this project, how did you define the problem set? You know, there's a lot of threads to pull on. There's proteoforms, there's single protein sequencing, there's structural stuff, there's issues of mass spec, solving foundational issues of mass spec. How did you define your problem set and then figure out like, okay, these are problems that are tractable and these are things I can solve versus maybe things that are like, you know, we should probably, we need somebody else to work on them first before they become feasible.

Yeah, I mean, so maybe to narrow things down a little bit more, I'm specifically— at least I have been working mainly on mass spectrometry-based proteomics. And in that sense, sort of like as a data modality, I've been developing methods to get new insights to sort of characterize proteins based on the mass spec data, especially tandem mass spec data. And I guess that's been mainly by the virtue of Bill being an expert in that field and sort of like focusing on the problems in that space. I was already a newcomer, so I was more or less happy to go along with sort of like a narrow set of problems that I could more easily attack rather than sort of like doing the problem search from a much larger space. So in that sense, that was again, a bit of a happy coincidence that I sort of like stumbled upon aspect-based probabilomics. And I guess to maybe, give the listeners a framework of like how things are really working in that space. So really, the mass spec proteomics is the very first step that you usually take at this point to characterize proteins of any sample at scale. So before like any— you have any idea about the sequence information, let alone like structure or function, you have to sort of like identify and quantify the proteins that you're interested in in a given sample. So that can be a blood, that can be sort of like a sample from the ocean. It can really be anything, as proteins are the machines of life in very basic terms. So in that sense, with mass spec-based proteomics, you would collect some sample and you want to characterize the proteins in that sample. There are usually two different types of problems. There are identification problems, there are quantification problems. I have mainly worked on identification problems and today I guess we'll mostly be focusing on that. But basically identification is characterizing those proteins in the sense that like identifying their amino acid sequence. And on the quantification end, you're basically trying to either absolute terms or in relative terms trying to count how many of those proteins are actually in your sample. On the identification front, so you have your proteins, you want to figure out their amino acid sequence. Sequence, the first thing that you, that you usually do is sort of like breaking down, breaking them down to peptides, like short, shorter protein fragments. And then those peptides basically go into the mass spectrometry instrument, the mass spectrometer. And in the instrument, the end goal is really measuring the mass of particular fragments of that peptide, of that initial peptide you started with. So to do that, you isolate the peptide and you further sort of like break it down by ionizing it into smaller fragments. So at that point you're like working with smaller chunks of amino acids that all came from the same peptide. And basically you sort of like measure the mass of those chunks, and at the end you get what we call a mass spectrum. And mass spectrum is basically a bunch of peaks that on the, on the x-axis you can think of as, as the mass of the particular chunk or the particular fragment, and on the y-axis you have a measure of their abundance. So how many of those sort of fragments that you counted. So that's the mass spectrum and with the identification. So the real goal is looking at that spectrum, you're trying to predict, or you're trying to identify which sort of like amino acid sequence and which peptide generated that sort of like in the first place. So that's in very broad terms, the identification problem. Usually, so people, Whenever they encounter this problem and thousands of sort of like proteomics experts and scientists do this every day and in all sorts of different experiments, you usually consult a database of sort of like peptides that's usually either from sort of like a library, like a spectral library that you collected from earlier experiments, but more commonly based on genomic sequencing that you sort of like sequence the genome and then infer the potential or possible proteins and peptides that could come from there. And basically, you match the observed spectrum to that database to basically identify which sequence generated it in the first place. That's the conventional approach. But the real problem that we have been targeting and I have mainly worked on so far is called de novo sequencing, in which case you don't have a database. And given a single spectrum, you're trying to predict which peptide generated it without relying on anything other than the spectrum itself. So that's the problem. And one might ask, like, why, while we have this conventional sequencing, this database sequencing approach, why would you need sequencing at all? Which sounds difficult for the person who's maybe hearing this for the first time. It really is more difficult than doing conventional database search strategy. But sometimes, whenever you basically don't know a priori the potential peptide sequences that might exist in your sample, you have to do it de novo. So that's the key idea. And that usually arises in cases where either you're not working with a model organism, or even if you're working with a model organism, there can be different enzymatic digestion strategies which change the change how your peptides look like. So usually there's like all sorts of different strategies, try to make that as uniform as possible to make the searching easier. And alternatively, if there are mutations, so if there are mutations, the peptides that you would infer from the genome would not sort of like correspond to the ones that you would actually observe. And a lot of sort of like important applications, especially in the context of like immunopeptidomics, antibody sequencing, there are cases that what you really care about is those sort of like mutated peptides and hence proteins that you can't really capture with the database. So in those cases generally, and like as I said, even if it's proteomics antibody sequencing and metaproteomics again, in which case maybe like species that you don't even know about in the sample, people usually resort to using genome sequencing to learn which sort of like peptides and proteins are actually in their sample. So again, to put this in context, this was like mass spec-based proteomics and like how we conceive of the identification problem there. And for anyone like who's working in the protein biology, that's very upstream of anything sort of like the other methods do. And I guess talking about like machine learning and protein biology probably doesn't go without sort of like mentioning AlphaFold or like some of the other protein language models. So all of those are downstream of what we're doing. We have— we first identify the sequence and then all of those models work on the sequence, model the sequences, and sort of like in our case, try to predict the structure and hence function. So yeah, that's a nice sort of like overview.

That makes a lot of sense. I think your work is really about like what actually proteins exist in the first place.

Yeah.

And I think proteomics spans everything from an ELISA just a simple amino assay to what you're doing, the novo mass spec, and everything in the middle. And so it's just kind of like what you're working on is really building tools and methods that like discover new proteins in biology. And then everything upstream, I think downstream, or I don't know, upstream or downstream. Downstream is like the analytics, but like there's also simpler tools out there than just the mass. I've used mass spec, it's really fun. But man, the variability and essentially you have to get this protein sample and it, you know, it's like I always imagine mass spec is like you're, the proteins are flying, you know, they're being ionized. They're flying in this machine.

That's a nice way to put it. Yeah, exactly.

And they're being like charged and, you know, the bigger ones, you know, won't travel as far and the small ones will. And then you try to figure out the spectra. Is there actually really complicated? I've done mass spec a lot and Gets pretty complicated after a while. You definitely need like a— Berkeley has this guy Tony. He's like a wizard of mass spec. I don't know Washington's mass spec facility, but in Berkeley, the mass spec facility is in a basement and it's real kind of creepy. It's like going into a dungeon of protein measurement. So when you think about de novo sequencing of proteins, like, you know, you've alluded to why it's important, right? You can discover proteins that have never been measured before, then that could be really useful, especially for like antibodies. Yeah, really valuable actually. If you find a rare antibody that's never been discovered that has like, you know, a therapeutic effect, yeah, it could be worthwhile. I think really, really useful. What are the— when you work on this project and the paper you published, what was kind of the challenges to sequence a protein that's never been sequenced before? Was it on the analytics side? Was it on the instrument side or just a combination of factors that had to be solved?

Yeah, maybe sort of like before trying to address some of those challenges, I can give you an overview of the model itself as well. So basically, so here's the problem. Here's the sequencing problem. And I guess in the past 5 to maybe 5 to like 6 years, people have been trying to solve this with machine learning. As pretty much with anything else like in science these days. And they have made some great progress both with like sort of traditional machine learning algorithms and sort of like more recently with deep learning algorithms. They have made great progress. And ours is basically another sort of like model in that line, which sort of like pushes the envelope. And really we conceived this, you know, sequencing problem in the same way that some of the, as I alluded earlier, sort of like some of the defining technologies or defining methods in natural language processing. And so like language-based applications of machine learning have done in the sense that if I was to sort of like drive an analogy between language processing and sort of like mass spectrometry-based proteomics in the, for example, translation problem, which is like one of the important problems in NLP, you would basically have two different languages. In one language, you're like operating at the level of sentences, and each sentence consists of words. So it's a sentence is a sequence, and you're sort of like words are tokens in that sequence, and you're basically doing this translation between, um, one sequence to another. We usually refer to this as a sequence-to-sequence model. We basically were the first ones to conceive the sequencing problem as a sequence-to-sequence translation problem. And solve it with the same sort of architecture, the same approach that NLP literature employs, namely the transformer and self-attention mechanism. So in our case, we treat the mass spectrum as a sequence itself, although it's unordered. For the purpose of this analogy, imagine as a sequence. And so basically the peaks that I mentioned before corresponding to individual fragments are tokens in that sequence on one hand. So that's what we have on input. And on the output, you basically have protein— oh, sorry, the peptide itself, again a sequence, and the tokens are the amino acids. And you're translating between, like, these sequence, uh, this sequence on the input, uh, to the, to the other one in the output. So that's basically how the model functions.

Um, and I'm not the expert here. I have a friend who's really good at this, like, talk to her pretty frequently. But this is kind of the same model as, like, ChatGPT in a lot of ways, where, uh, or maybe not In terms of using a transformer model to convert.

Yeah, exactly. In terms of using a transformer, it exactly is very similar, conceptually very similar. The only difference is in the ChatGPT, it's a generative model and you have the encoder. This has two parts. On one hand, you're encoding the input and on the other hand, you're decoding the output. You're encoding the mass spectra and decoding the peptide sequence. In the ChatGPT's case, you are training this model. You're only training first encoder to learn representations of of the language in that case. You're doing something very similar for the mass spec. And in their case, they're basically training this whole thing using the language modeling paradigm where you don't have labels and you're using the language itself and basically something like a next token prediction task to sort of train the whole model. Whereas in our case, we have access to labels. So we basically have mass spectrum and peptide pairs. That we derive from like the database search strategy. And using those pairs with this sort of like supervised learning paradigm, you're teaching model to map the— or translate the mass spectrum to the peptide sequences.

Yeah, I think kind of similar, maybe underlying architecture, but then you have better annotated data and more focused. And so, you know, when you think about like, I don't know, Seems like you were early in using these models, like protein sequencing. Like, how do you, how would you kind of think about maybe in general, like using machine learning models in bio? What are kind of the prerequisites to like be productive? Is it just kind of annotated data or is there kind of other things that like are necessary to kind of like kind of make an impact in biology within machine learning?

Yeah, I mean, I don't think there's a simple answer to that, but maybe to give an example of like a needed direction for like annotated data subject, I think there have been like really great examples in the protein language modeling literature that has shown basically without any labels, just using the protein sequences, you can capture some of the evolutionary dynamics of the proteins. And use that to do all sorts of great things downstream, including structure prediction. So there's been great literature on that front. But also on the other hand, for example, for regulatory genomics, similar approaches do not work as well because of the complexity or sort of like really the characteristics of the DNA and the sort of like the sequences of nucleotides rather than sequences of amino acids that still require strong supervision to be sort of successful. To be actually useful. And the best methods, or like the state-of-the-art methods on that front, have been more supervised than sort of like unsupervised, as is in the case for protein language models. So I guess the high-level sort of like takeaway from that could be still in biology, there are like no one-size-fits-all answers or methods or paradigms that solve like every question. And I think really the excitement and the intellectual sort of driver of all of this is you have to understand the domain well. You have to understand the constraints of biology and really sort of like bring in these shiny, cool methods from other domains and be mindful of the parameters that you're working with and sort of like whatever problem you're trying to solve. And I think that's also like really exciting and really sort of intellectually stimulating for researchers in comp bio and sort of like biology in general.

Yeah, I'm sure like throughout this whole process of building up the model, like how many people did you talk to? How did you get feedback? Were you like, like, uh, did you have to go to various professors or something? I think a lot of people in biology think of the comp bio people as like stuck in the basement, stuck in their apartment, or just coding all day. But maybe you're alluding to something different where a lot of it's collaborative. Where you're coding, you're talking, and you're trying to get feedback to make sure you're doing something that's useful and not so much just like— what's the pejorative biologists always call the comp bio people? Like armchair biologists. They're not really doing the work. You're not in lab. When you build the models, how did you get feedback on what you were doing? To make sure like it wasn't just like inaccurate.

Yeah, I mean, being in the lab, being in a wet lab is of course a different experience. And so being a computational scientist is a whole other experience. They both have like their challenging bits and maybe sort of fulfilling bits. On in terms of like our methods and probably for any method in ComPy or any like any sanely developed method, I think you can't go without like actually consulting people who would use it as end users who understand the data collection process and who understand, who have a better sense of, who have better intuitions about the types of problems that you're trying to address with that method. I think we had like a lot of different great streams of input in that sense, both within department but sort of also from elsewhere. And I guess more recently, as our method has been sort of like out there and we have been like putting really great effort into making it as usable as possible. People have been using it, people have been providing us great feedback, and people have really sort of like tried to tackle problems that we haven't even really imagined to begin with. Some are really niche problems. And again, these are people that I personally, or like even my advisor, has never really heard of. And well, they're really trying to sort of like do cool things with what we have developed. And as a junior scientist, I think It doesn't happen every day, but that's one of the things that sort of like give me the most pleasure of doing all of this.

That's cool. I think when you invent something and all these other people are using it, like, wow, like, that's, that's pretty meaningful. And you get those emails. So you build up your model. How did you go about testing it? How'd you go about like, like, do— did you do the experiments? Were you in a lab? Or how did you, how did you go about like making sure, okay, create this model to do de novo protein sequencing. Does it work? How did you kind of go about verifying it?

Yeah, that's, I mean, that's one of the most important things in machine learning. Like, once you build this full model, like, does it actually work or is it actually useful? I think those, even those two could be like different questions. Like, does it work? Or like, is it, is this useful?

Yeah.

And so in our case, initially, and like with our ICML paper, We were basically comparing this method with a host of sort of like existing DNA sequencing methods on an existing benchmark. So again, I think one of the main things that drove this much progress in deep learning and machine learning in general was the existence of like really good benchmarks that people painstakingly put together and like compile in, not only in CompBio, but maybe like more conventionally in computer vision and like in NLP. So in our case as well, that was like the go-to method or the go-to approach to test things out. So that has sort of like to some extent proven that it actually does work. But once you have that, basically with our follow-up paper, we have improved the model a little bit, but maybe more importantly, we have shown that like for some meaningful downstream applications, the ones that I mentioned before, like immunopeptidomics, metaproteomics, some really key applications that— does our model actually move the needle? Like, does it make some of the previously, previously sort of unavailable things or previously inachievable things now more feasible? And we have sort of like, we have included all of those results in our recent preprint. Hopefully, sort of like, it's getting reviewed. We might be able to get it published sometime later this year. But beyond that, as I mentioned earlier, people also have been using it for all sorts of different applications. And there has been some external validation and like some, some other work that's basically shown, for example, for antibody sequencing, that our method is sort of like really, really changing things and really showing significant improvement over the existing, existing approaches. Cool.

Okay, so you get the preprint published initially, do some talks. How was that like, you know, like kind of the— how was kind of like, uh, getting attention for your work? How did you deal with that? How do you know And how did you after that figure out next steps? I think often people publish something and then you have like next steps. There's always that next steps section of any grant or paper and you just kind of make it up to just get the paper done. But then when it comes to like actually trying to figure it out, it's like, well, we can do this, we can do that, we can do antibody sequencing, we can do all this other stuff. We can make our method better. So like, how was that process of like, dealing with it, kind of going through that process of putting your work out and getting feedback? And then how did you then figure out what's the best use of my time? You know, should I just do— should I work on a Dune project? Should I extend this project? Should I find an application? Especially for tool builders, that's like really hard.

Yeah.

Like if you're doing like genetics or a new pathway, these questions aren't important because like you just focus on your narrow area of biology. But if you invent a method, it's like you can do anything now. It's kind of, it's actually kind of, I've never been in that position, but seeing friends be in that position, it can be very stressful.

Because yeah, indeed, indeed. And I think, I think that's a great question. Like, as method developers, you, you always have to think about, or like, do I, do I actually continue building this up? Or should I sort of like set on a new direction, maybe a more promising direction that can change things a bit more radically than incrementally improving this existing method? I think that's an important bind. That's a bind that I'm still facing like to this day, maybe every day, as I try to prioritize like different streams of work. That usually comes in the form of like working on maintaining or like even developing our existing tool Casanova versus like working on some of the new directions. I think one of the most important things there is, I mean, first of all, trying to, if the method has like some traction, trying to like after the initial investment in terms of time and effort, trying to keep up with that to some extent, like trying to really provide for the user base because eventually like as method developers, you really want to see our methods being used to discover new biology, to discover sort of like new science. That's the bottom line. But also in some sense, especially as a PhD student, you're also sort of like getting trained. You need to publish, you need to continue to sort of like show that you're capable of discovering like these new ideas and following up on them. So in that sense, I'm trying to strike a balance between the two by sort of like both pursuing one on one hand, again, with the help of my collaborators and sort of like working on new directions on the other hand. But again, it's, it's really sort of like, it can be a really tough problem to balance off.

It's hard, I mean, honestly, I don't— I think it's— we're probably still trying to figure it out. I think the best person who's, who's like been in this spot, uh, I should do a podcast with him, a guy named Bart. Uh, Bart, I can't pronounce his last name, but he invented DeepChem, uh, in Vijay Pandy's lab at Stanford, and he did a really good job of building a community around his method. Kind of put a GitHub repo out there. I think even after grad school, he continually kind of refines it, updates it, keeps it usable. And then over time, the number of users of his package grew and it's become like a really awesome, like, drug development tool. And, you know, he's not really working on it full time. Like, but, but other people have extended his work and it's actually Like sometimes, especially your professors, I know you get kind of pigeonholed into your method. So you become the method person. I have this method and I work on this, have this one tool and I do everything with it. And that can limit you creatively, creatively is where you're like, oh, I'm just a de novo protein sequencing guy. And then people come to you for it and like, awesome, but I can do other things too. You know, I can, you know, build other tools.

So just kind of— Yeah, you, you have to show that like you're, um, you're not a one-trick pony and you have to sort of like diversify a little bit as well.

It's really hard because you have success in one place. It's hard in general to have success in one place but then not be limited by it and be able to grow and trying to find a balance. I think ultimately what you're doing now is trying to get other people to use your tool. That's probably the best way to do it because then they can extend it themselves and then it becomes like ecosystem and have other people do the work. And then you can spend more time inventing new things and be useful to more people. And so, OK, you get the paper published, you're working in grad school. Kind of anything else you're working on right now? Is it anything non-de novo protein sequencing related you're exploring? Maybe on the foundational model side?

Yeah, that's actually something that I've been really sort of like— that's keeping me busy nowadays. Thinking about some other problems in that space. And of course, there are some other things as well, but on the research end, especially on the programmatic front and also slightly outside it, I have been really looking into ways of incorporating some of that foundation modeling ideas from NLP and from elsewhere into this domain. Because as I mentioned earlier, what we are doing is basically having this supervised setup to teach a model based on existing peptide annotations how to de novo sequence. But at the end of the day, usually in cases where you need the de novo sequencing most is the spectra that you can't successfully and confidently get peptide annotations using the database search strategy. So being able to sequence those would be really, really an important boost to this whole identification problem. And to be able to do that, I mean, you would ideally have to have a way of training your model with that. Spectra, which requires some sort of self-supervised training. And this really ties us into the foundation modeling idea. So currently, I'm exploring ways of first training the model in a self-supervised way before doing some fine-tuning on this particular genome sequencing problem. And such a model, if we successfully do this, it would also be able to address some of the other downstream applications of proteomics. So not only genome sequencing, but other other problems like chimeracy prediction, other problems like modification prediction. So basically anything, like any problem that you can think of that takes spectrum as input could benefit from like a foundation model, which only trains to model sort of like without any labels, any peptides in sight, only is trained to model the spectrum in a similar fashion to sort of like what the language models are doing in other domains, in both the proteins or like with the natural language.

Cool. Maybe related to this, how do you see the endgame of proteomics? Do you see it similar to sequencing where every protein will be sequenced by mass spec or something similar to that? Or do you see it more of like there's going to be different options to sequence proteins? Maybe for things we already know pretty well, we just use an immunoassay, and then for things we don't know, we use some sort of mass spec method. What's kind of, let me, 10 years from now, How do you think, like, protein sequencing evolves, especially in terms of, like, the tools we use right now? Proteins, people, a lot of people don't— a lot of, honestly, a lot of academics, the way they measure proteins is Western blots.

Yeah, that is also true.

Yeah, you know what I mean? Just use a Western blot. Why go through all this hassle of mass spec and the money you spend? Just make a gel, do a Western blot. Is it there or not? You know, how do you think protein— do you think proteins— how do you think protein evolves? Do you think Western blots go away, or do you think it just kind of— we kind of— these older tools stick around, and then on the high end, we just get better and better at finding new things?

Yeah, I mean, from where we stand right now, based on, um, based on my sort of My impression from some of, like, some of the discussions that our colleagues are actually working with these tools, actually working with mass spectrometers, are having— my impression is that most of those are here to stay, including mass spec. I mean, right now the, the great debate in the field is mass spec versus some of those, like, single, single molecule sequencing methods, or like really methods that don't rely on mass spec and really, like, like nanopore sequencing or some of the other methods directly try to identify the proteins as is and their sequences. It seems like, yeah, mass spec is here to stay, both by its virtue of scale and relative cheapness, maybe I should say, compared to some of these up-and-coming methods. But also, one could imagine it will also potentially become easier and easier to utilize And over time, at some point, it will have its sort of like genomic sequencing moment. But it's like much more standardized, a lot cheaper, addresses single cell, addresses like those different setups that can be used. And over time, the traditional methods sort of like, especially at applications that require some scale, it will be on its way out. But again, mass spec will be here to stay. And as sort of like mass spec continues to dominate proteomics, computational tools like ours will be sort of like continue to be really relevant and important, leading to like new discoveries.

Yeah, I remember doing LC-MS in college, and then 10 years from now, then it's just like the sensitivity of these devices have gotten a lot better over a decade, and then the interplay between like computational methods too has helped kind of magnify that, the instrument improvement. Uh, yeah, I'm just curious, I think proteomics kind of evolves differently Like in sequencing, you saw like people were sequencing things in gels for like a decade or two, like these massive gels and you had to sequence DNA. And then we made the transition to microarrays and then sequencing by synthesis. And before you know it, like, you know, most people just send things off to the sequencing core. And I think that kind of coincided with like the number of like viable leads that sequencing provides. I think sometimes in proteomics, you could pay for a mass spec or do a panel, but why in the first place? I think that's kind of the thing. I think that not only the tool side, but the why, it's like the application, like the meaning of whether a protein is overexpressed or not, or certain proteoforms. I think trying to figure out like the tools will get better, but also like the applications, the value of spending the money to not do a Western blot, you know, or even do an immunostaining to get better kind of interplay. And maybe that's another conversation around like proteomic products.

Yeah. Mm-hmm.

Also too, maybe in general, how do you see the impact of like, you know, I don't know what to call it, AI in general, but there's generative AI is a big term now. But you know, there'll be a new model for your schnauzer. How do you see the impact of like AI on biology? I think there's some people who want to make like predictive models, like predict everything. I don't know if that's possible, you know, that's, that'd be cool though, but I don't know if that's possible. Or how do you see AI impacting biology in general? The one thing too is like ChatGPT is pretty useful. I thought it was a little skeptical initially, but it's actually, I can give it scientific questions and it gives me good leads. So that's one end too. But yeah, all this progress helping science, helping biology.

I mean, I'm obviously like based on, based on what I'm by the virtue of what I'm working on right now, I'm a big proponent of AI for science and AI for biology in general. I mean, there have been like many false starts or there have been sort of like false, really sort of like false improvements or sort of like non-significant improvements that we mistook for sort of like real progress. But at the end of the day, I think the tools really contribute to our pursuit of understanding life. And maybe in terms of specific applications, and as you mentioned, in the context of language models and some of these new chat interfaces, one thing that I've been recently thinking about, and of course, I think to some extent dates back to Facebook AI Research Galactica model, which not only functions as a chat, sort of like a chatbot or like a chat interface, like the language models that we are more familiar with, but also is able to process and really sort of like meaningfully interpret some of the biological modalities like protein sequences, like compounds, chemical compounds, like file strings to be able to like receive queries in that format as well. I think like new models, new language models, projects which take that direction really can achieve two different things. I think like two different end goals could be possible. On one hand, the simpler one, you can end up with like a great interface that scientists can interact with to like put in all sorts of different queries to basically make them like, make it easier for them to discover new science. That would be cool. Even if it's just like a facilitator and doesn't build on top of our current understanding of biology or current state of the art of all of these like different bespoke models, like specialized models to specific tasks, you can take like, you know, sequencing as one of them. But on the other hand, like as a second goal, if training these models, like these large models in the context of everything that surrounds them, so given a protein sequence that would be like all of the knowledge that, like all of the textual knowledge that surrounds it, but not only the textual knowledge, but more broadly, like an understanding of chemistry, of biochemistry and how sort of like, for example, how mass spec data relates to proteins, like having a single model that is able to make sense of it in some sense, like that's able to extract and represent all of that information. And in a way be multimodal. Or maybe not multimodal, but like omnimodal, be able to process all of that sorts of model input. Such a model could actually be more powerful than like any specialized model alone. So that such a model, for example, could be more powerful than our tool, like the state-of-the-art sequencing method, and be also able to do all sorts of other things and meanwhile provide like a great interface for scientists. If we go anywhere near that, I mean, so far we have made significant progress, I think, with Galactica and maybe similar methods on the first problem of having this easier interface. But on the second problem, I think we have to prove that bringing everything together actually improves the outcomes and our downstream predictive power. But if that happens, I really would love to see it. Sort of like that would take us.

Yeah, that makes a lot of sense. I think like, you know, AI has solved very focused problems in biology, maybe protein folding, your work with protein identification, things like that. And then more broadly, maybe it helps provide leads in the, you know, taking all this information and be able to like make sense of it all. Not make sense of it, but provide potential outcomes or leads that you can assess yourself. Maybe it helps specialists like peer backwards and kind of put their work in context of everyone else's work pretty efficiently. And maybe it, you know, I think it hopefully makes science like more efficient and creative so you don't get like, you know, I think if you're a great scientist in general, you're a specialist. You know something really deeply, and sometimes you can get stuck there. So hopefully new tools can like help people you put everything in context. And so you're not missing out on things or you're not like overlooking things. And maybe it helps. I mean, that's— grad students have to read— how many papers should you read per day or per week? One question, when I talk to grad students, I had a friend of mine, he's like, yeah, I don't read papers, just read the abstracts to keep up to date. How do you keep up to date with everything? Do you just read abstracts or talk to friends?

I guess I don't like— I don't keep up with things like in general. But yeah, I mean, I would really commiserate with anyone who feels the same way. And I would say that like in my third year now, I have a slightly better habit of like reading papers on a regular basis, both on the bio and like also on the machine learning end. And more broadly, like broadly across the fields, reading the abstracts and like really sort of like over time, having the sense of like what you need to extract from a paper becomes really important. But for myself, I think what really panned out was having like a dedicated time of the week or a day. It's usually at the level of a week that you're like reading papers. And you're normally, I mean, people are really good at like most researchers are really good at hoarding papers. So you hoard the papers, you have a way of prioritizing them. And when that dedicated time comes, like a few, maybe a few hours a week, you just like, no matter what, you read those papers.

I have a Google Doc of pages of links, papers that are, that are interesting on Levera. It's kind of a, it's very much a bummer. I try to read a paper a week at least and really go deep and like, okay, let me, not, not for the knowledge, but just more for like, I find reading papers is useful to like understand how things are done. Like read the methods and, and kind of just really dig deep rather than trying to, you know, reading the literature is too noisy. But if you read one really well done paper, yours, and then you could really understand like not only the discovery, but then how, you know, how to— how you actually get there in the first place. But maybe transition. Okay, you're in grad school. What's the game plan? Do you want to become a professor? You've also been corrupted with the entrepreneurial thing, so we talk about that.

That's right.

Yeah, you know, I don't know, are people mad at you for, you know—

I'm joking.

But, uh, what's kind of that— what are you kind of, uh, your, your G3, third year you know, you're going to get your PhD sooner or later, but how do you think about where you want to spend your time, your goals, you know, and yeah, things like that?

Yeah, I mean, I really, I really come to grad school thinking I have to sort of like I have to do research for a living after I leave here and like all of this training will work towards that. And that remains my mantra to this day. What that could look like, I don't know, that could be a little different, but in my mind I have this sort of like maybe plan A of trying to continue to do publication-oriented research in some of the machine learning or AI labs in the industry, potentially at a tech company or like an affiliate, or similarly, like more recently, pharma companies are also making great strides on that end. So that would be ideal, like continuing to publish continuing to focus on really like academic science outside of academia. That would be the ideal sort of outcome that I can see myself doing. At least like in the next 5 years after I graduate, which is hopefully not too far off. As an alternative, I mean, I'll mention the entrepreneurship in a bit, but like generally there are all sorts of great biotech companies like biotech startups who are not exclusively publication-oriented. They're more product-oriented by the nature of sort of like their existence, but they're also doing like really cool cutting-edge work. So that would be another way to sort of like stay engaged with science, continue to do science, although for different ends. That's probably a plan B at this point, but still something that I'm like really considering for even for the short term. But more broadly, maybe to to gaze beyond like 5, 10, 15 years. I mean, I've been fascinated by entrepreneurship, I don't know since when, but someday I would love to start a company and sort of like really pursue a venture in that sense. And like even right now, especially like starting last year, I have really dedicated a significant portion of my time to explore that space while still in grad school. That was namely getting involved with Nucleate. So Nucleate is this nonprofit really sort of like targeting academic trainees in biotech who want to develop their own technologies or who want to commercialize their technologies that they sort of like already developed in some academic context. Doing that, doing all of that for free, all of that like in a nonprofit context. And basically, yeah, I got involved with Nucleate. I basically founded the Seattle chapter. They didn't have any sort of like local presence. And we basically set up a team to like organize everything in this region. And the end goal is sort of like, the end goal has been really running this program called Activator, which sort of like functions similar to an incubator and like an entrepreneurship training program targeting the local PhD students and postdocs who want to commercialize technology and who want to sort of like want to start something up and spin off their labs to address a challenge in human health. So that usually looks like very different things that could be like a diagnostic tool, that could be like a therapeutic, even like digital health applications and computational tools have been really great examples of companies. And in that context, in Seattle and like more broadly in Pacific Northwest, we have been supporting our initial batch of companies, initial batch of teams, who are pursuing this goal. They're all great. And in a month or so, we will sort of like be graduating them and hopefully they will have made sort of like most of the program by interacting with the mentors, by sort of like going through different workshops to really address like important challenges that every scientist or every junior scientist asks themselves as they sort of like thinking about, as they start thinking about how I can turn my science, my sort of like my inventions into a product. A viable product and build a venture around it. So in that sense, it's been really great to get involved with Nucleate and like really lead an initiative here locally. Beyond like being a— giving me some firsthand experience with the venture space, I think it was also really important to interact with like all these different kinds of people. I'm not even talking about like investors and people from the industry that we get to interact. That's great. Like it's a great networking opportunity, But beyond that, I mean, normally I don't really see any medium a PhD student would like sit on the same table as an MBA student or a medical student. So in that sense, you really get exposed to all of these different paths, all of these different perspectives. And the catch is all of these people are like really excited about biotech and similar things that you're excited about. So in that sense, it was really great to be building something together with my teammates, both here locally, but also sort of like globally from the Nucleate network.

Yeah, and Nucleate's really awesome. I think great work to build up the Seattle franchise. I'm a mentor to Nucleate. And so as an investor interacting with people in Nucleate, I think the key thing is like the socialization part you're alluding to, which is like, you know, especially in grad school, my PhD programs, you can be kind of isolated to your lab, to your building, and creating a a layer where people of similar interests in terms of impact and work, rather than maybe exactly the projects they work on, but more like what they actually want to do after, or they would— what they want to do with their work is make and build a company. Getting those people in the same room, sometimes, especially universities, scientists are in the same room based on like a talk, a scientific talk, uh, things like that. And so that's how you meet people by like technical interests, but to create a setting where people are gathered based on like a business interest or interest around startups is pretty valuable because it gets people who wouldn't talk or people who maybe interact, like, you know, MBAs aren't going to, you know, talks about your work, to be honest, but they'll go to like a Nucleate event, hang out, and then, if you can create enough of those interactions, good things happen, right? You plant those seeds and the nucleus creates a very valuable community and network so that then kind of build that, that a foundation for a lot of science. So like, if you've been doing this, it's been a lot of work probably. Uh, what's been your, like, how did you, how are you learning about business entrepreneurship? I think the key thing as an inventor Always the kind of the hard part is then you invent something, then how do you value your invention? And how do you know something's valuable or not? There's always these tropes of scientists who get screwed over in business, you know, because they don't know how valuable what they have. How did you— what kind of skill sets, what kind of resources, or maybe who do you talk to or lean on to kind of learn the lingo of business, kind of learn the terms, learn how to analyze markets, and ultimately, how do you build up the kind of the repertoire to value your work? And then also more broadly, when you interact with scientists.

Yeah, I mean, I think, as I mentioned, I've been invested. I've been interested in sort of like getting involved with entrepreneurship and the venture space like more broadly, even from beforehand. But at grad school, I found myself sort of, I basically found it hard to engage with those topics like from basically like reading on my own and like doing stuff on my own. And in that sense, Nuc8 really gave me some structure to like pursue those and really sort of like people usually learn by interacting with others and I'm no different in that sense and really getting some exposure both to other people, like other leadership team members in the NPH context who are more experienced about these subjects, but also like through interacting with the teams, with the startup teams, like in our cohort and trying to address their needs, you find yourself in like a more relevant context to learn all of these different questions surrounding both like both the business and, but also some of the more applied scientific questions. That become really relevant in the context of building a venture. So I guess my short answer is through exposure, through exposure to people, and making this a part of your life that you're actually compelled to learn about, that you're compelled to retain what you learn, even if that involves— that still involves an important amount of reading, important amount of listening to other people, making it a part of your life. I think that's a key point. And beyond entrepreneurship and venture building, I think that's relevant to anything. If you manage to make it a part of your life, that's, uh, probably the best way of like learning about it and retaining in the long term.

Good point. I think, yeah, just finding people to talk to about it who are experts in it and then be able to like make it a habit. That's the harder part, making the habit part. That's the harder thing. Yeah, make it a habit because like you're so overwhelmed sometimes of like, sometimes it's like you're so tired you don't want to go to that networking event. You need to make it a habit, you know what I mean? I totally— sometimes you're like, oh, I just want to sleep, but maybe I should just go meet these people, you know? Even today I get like those feelings where it's like, I just want to go for a walk, but maybe I should go to that business party because who knows, serendipity might happen. And it has. So, you know, kind of always putting yourself out there and, you know, getting exposure is really important.

Exactly, exactly. And I mean Nucleate has been something that I've been mostly doing on the side, so it's more or less like a part-time pursuit. But that, that there have been like weeks, um, and weeks on end that I had to spend like 20, 30 hours on it while still trying to do research, still trying to make progress. So I can, yeah, I can really, um, test the importance of like sticking to it, uh, being persistent and maintaining those connections.

Yeah, I totally agree. But okay, cool, this has been awesome. We'll do a follow-up 2 years from now and you'll be a professor, have a startup, I'll invest in it. You'll be a professor. One thing too, I have a lot of respect for people with your background. I have friends like you, just like people who are world-class engineers in biology. I have a lot of respect for the conviction you have. It takes a lot of conviction to be like, this is the problem I want to work on because there's so much optionality. You can work on anything essentially. And then it's like right now it's saying maybe transformer models and chat-based interface products are getting a lot of hype. To be able to stick to your project and conviction is like a lot of respect. It's really hard. At least for me, it would be really hard to not have to kind of just continue to keep making progress. And when I'm— just like I said, just to be an engineer in biology, you have so much options. One of the harder things to deal with, I think, psychologically, but to keep at it and not— I saw this a lot with my crypto friends. I have friends who are really good at AI and they jump into crypto. Bad decision. Right. And so you want to keep conviction in something is like, indeed.

Yeah.

I got to really popular.

You got to stick to your guns and yeah, I'm really, I'm happy to be in this field. And I mean, if the others in some of those other domains that now see like now seem more shiny, more popular and maybe more cutting edge. I mean, if they make real progress, hopefully someday we hope to have them here over on this side of the aisle. To solve some of the important problems that affect us all. Awesome.

Well, Mila, thanks for doing this. I really appreciate it.

Likewise. Thanks for having me.