#36 Data Science and Ecology

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey.

On datacamp.com in this episode of Data Framed, a DataCamp podcast, I'll be speaking with Christy Bally, Assistant professor at Kent State University, about data science, ecology and the adoption of techniques such as machine learning in academic research at large. What are the biggest challenges in ecology that data science can help to solve? What does the intersection of open science and data science look like in scientific research? What is happening at the interface between data science and machine learning methods, which are pattern based, and traditional research methods, which are classically hypothetical deductive? Is there a paradigm shift occurring here? Oof. One of the reasons I love this convo with Christy is that she is wonderful at placing the work that we do in a cultural, historical and sociological perspective. A task that we're not often incentivized to do. Listen on listeners to hear how the modern scientific method, in its hypothesis driven, statistically tested form, emerged from us living in the small sample size limit for so long, and how the way modern science is done, how knowledge is created, is inextricably linked to the way work is incentivized and produced in late stage capitalism. Knowledge isn't created in a vacuum and our science is a function of our culture. There are many ways to describe the world and pattern based data science methodologies are on the rise in basic science research. It's cultural, it's generational, it's wild, and it's worth discussing discursively and at length. Now that's enough out of this 6 foot 4 bearded Australian. It's time to hear from the expert. I'm Hugo Baun Anderson, a Data Scientist at DataCamp and this is Dataframed. Welcome to DataFrame, a weekly datacamp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host Hugo Bown Anderson. You can follow me on Twitter hugobown and datacamp@datacamp. You can find all our episodes and show notes@datacamp.com community podcast before we get into the chat with Christy, I wanted to let you know that I've written an essay for Harvard Business Review synthesizing what I've learned from my conversations with all the exceptional guests on this podcast. If you search what Data Scientists do hbr, you'll find it. You can also find it at bit ly HBR hba. That's bit ly hbr hba. And I'll also include the link in the show notes. One last thing. This Wednesday, August 22nd, I'll be giving a webinar called what Managers need to Know About Machine Learning. In it, I'll tell you everything you've always wanted to know about machine learning but were too afraid to ask. After this webinar, you'll understand the basic concepts in machine learning, how they can apply to business problems, when to use them, when not to use them, and how to talk the talk with data scientists. Sign up for it at bit. Lee HBawebinar that's bit Lee HBawebinar. Hi there Christy, and welcome to Data Framed.

Thanks for having me, Hugo.

It's a real pleasure to have you on the show. So we're here today to talk about data science, ecology, open science, what the data deluge really means for working scientists and data scientists. But before we get into all of that, I'd love to find out a bit about you. Can you tell us what you do?

Okay, so I am an assistant professor at Kent State University. I am coming up to my one year Professor Versary, I guess.

Congratulations.

Well, thank you very much. I started August 20th last year, but before that I was a postdoc at Michigan State and a fellow at Mozilla. So I'm interested in long term patterns in ecosystems, so how ecosystems are behaving and how their processes are unfolding over time. And I mainly look at this through the lens of insect ecology, so particularly their population dynamics. So the insects that I'm most interested in are the ones that do things in the ecosystem. So I do a lot of work with predators, so insects that are involved in pest control and pollinators, insects like bees that pollinate plants and so help with food production. I mostly work in human managed systems, so agriculture, park systems, managed forest, urban landscapes, anywhere where humans are interacting with the environment and need to get things back out of the environment.

Fantastic. And so just for clarity, for me, ecology as a general discipline is about the study of these types of ecosystems.

Well, ecology is the study of any ecosystem. So anywhere where there is something living you can study ecology and so Ecology is of course, not just about the living system, like the living part of the system. It's also about the environment that the living things are existing in. So there's a big part of ecology that's devoted to biogeochemistry, for instance. So that is the chemicals that, the chemical processes in the environments that are affecting the organisms and that sort of thing. But I'm mostly into organismal ecology, so I focus on the living things.

Great. And when you said you're interested in long term patterns, this type of, you know, looking at patterns, pattern recognition, identifying patterns, this sounds ripe for kind of all the tools that data science has developed in the past decades.

It really is. So I kind of accidentally stumbled into long term ecology. I ended up with a postdoc at the Long Term Ecological Research Site at Michigan State University, Kellogg Biological Station. So it's in southwestern Michigan. And I hadn't even considered that it was possible to do ecology this way before I arrived. And suddenly I looked at the data that they had been amassing since the late 80s at this one site where they'd been sort of intensively sampling the landscape. And it just kind of clicked with me that, you know, it wasn't just going out in an ecosystem that we could study it, we could also study it from the signature it had left in the data that we'd been collecting about it over time. And so I thought that that was really exciting. So my background, I actually started out in physics. My undergrad was in physics. And it, well, it seemed like a good idea at the time, but I got my first lab job in a biology lab and I'd sort of switched back over. And so I had become sort of a field ecologist in earnest. And working in long term ecology allowed me to apply a lot of the math and modeling that I'd learned in physics to biological systems.

Great. And so what happened then in your career in order to get you really interested in data science?

So a big part of it was the fact that I was the biologist who had the physics background. And so people would always ask me for help with their models and their math and their stats. And so people would send me data sets and say, hey, could you help me run an analysis on this? And I started realizing that the human element in data science was, well, the human element in data in general was problematic. When in physics, most of the data that you get is created by machines and so it's automatically machine readable. But when you have biologists creating data, you have the human element and everyone sort of hacking out their own way of handling the data. So they open a spreadsheet and they design their own way of entering the data, and that's how they think of the data. They think of the data more as a spreadsheet, more as a lab notebook than as a way of recording data. Data for prosperity, for future applications.

And there's not even necessarily a system of best practices around this.

Yeah, exactly.

Entry task. Right.

And especially with biologists. Biologists are sort of taught to think in the ecosystem, and then they're kind of let go and say the people set them forth on their data and say, okay, well do this. And they don't really fill that bridge. And so I noticed that there's a vast amount of data in biology and it's very inconsistently kept. And I saw this as an opportunity because think of how much more we could learn about ecosystems if we could really effectively compare the notes of biologists, like, bring together all their data in a meaningful way, rather than sort of stacking a pile of notebooks together, like making it a synthesis rather than collection.

Exactly. And so then my recollection is you started your blog, then you started working with data carpentry and software carpentry, right?

Yes, exactly. So I noticed that there were some trends in how people were handling data, and I admit some of them got me a little bit annoyed and felt made me feel heated about it. So I decided, well, this is an opportunity to create a blog where I can teach biologists how to better handle their data so that computers can read it better, so that other people can read it better. And at this time, I was sort of operating on my own. I had no idea there was a community. And it was actually data carpentry and software carpentry that found me. So Greg Wilson reached out and said, hey, I think we've got some ideas in common, when he started reading my blog. And that's how I got pulled into the community. So Tracy Teal, who is executive director of the carpentries now, she was at Michigan State at the time while I was there. And so she took me for coffee and she explained what this stuff was, and I went, whoa, there is a community around this? I had no idea. And so I attended my first hackathon where we would developed lesson material for data carpentry, and the rest is kind of history. I just started really embracing the community and started doing a lot more outreach, got involved with Mozilla, and now I am as much a data scientist as an ecologist.

Fantastic. And my recollection is that Tracey Teal also has worked in ecology. Is that right?

Yes. Yeah. So, kind of funny thing, we were both partially appointed to a project at Michigan State called the Great Lakes Bioenergy Research Project, where we were looking at ecosystem responses to biomass crops that could be used in bioenergy production. So she was more on the microbial end of things and I was looking at the insect population. So we were in different labs, so we didn't know each other.

So. And just on a side note, I actually taught my first data carpentry workshop last weekend at Cold Spring Harbor Labs with Jason Williams there, which was really exciting. It was cloud computing for genomics in R and some machine learning in R stuff that was a real treat. Yeah. So moving on, a decade ago, Wired published an article by Chris Anderson that was titled the End of Theory. Relatively provocative, the End of theory. The data deluge makes the scientific method obsolete. So in your mind, is this title correct or will it be correct?

So I love this article because I can use it to provoke so much conversation with people. So it's really nice because it was 10 years ago that this was published and so we know that this has not come true. So there are still theoreticians out there. So theory isn't dead yet and the scientific method is going strong. But the data deluge is changing how we look at science. So the scientific method, as classically taught, comes from this mindset of data scarcity. So from a small set of observations, can we observe the patterns that are predicted by theory? But data science turns this on its head. From vast data sources, can we use patterns to infer theory and mechanism or predict the behavior of a system from a classical standpoint, From a theoretical standpoint, data science is almost heretical. It is going at things completely backwards. But when you think about it, if you're trying to model something based on a small sliver of what can be known about the system, this is sort of what classical science is based on. You sample a population, but when your sample is basically the population, you don't have to guess what the whole the patterns of the whole population are. You can say this is what the patterns of the whole population are. And now let's go back and see if we can find the drivers for sure.

So essentially what's happening there is that we've been due to cultural reasons, essentially we believe, or we utilize this hypothetical deductive model of science, which essentially is formulating hypothesis that hopefully falsifiable, and then using experiments to test whether, whether it looks true. Or not. Right.

Yeah.

And culturally, we've accepted that method or been forced to use that method due to the scarcity of data. But I think as a culture, we've almost adopted the form of belief that this is the only way of doing science. Right?

Yeah, absolutely. So it's kind of funny, I think, that the hypothetical deductive model has actually affected how we write about science, how we perform science, to the point where we're almost trying to sort of shoehorn our way back into the hypothetical deductive model, even when what we're doing when our discoveries are not hypothetically deductively derived. So, for example, a lot of graduate student work is really, really exploratory. So the students necessarily have a hypothesis. They are just trying to sort of figure out how the system works. And so they go out and they make observations. And then there's sort of this pressure on the students to go and do a statistical test and show that there's some sort of statistical difference. And then you see students applying these post hoc hypotheses, but they write it in their papers as if they had the hypothesis at the beginning, because their advisors will write in the columns of their papers, well, where is your hypothesis? And the thing is, they didn't really. And that's okay. They went out and they discovered they observed a trend, and they don't necessarily have a hypothesis about it, but they are developing hypotheses. And so it's more of a hypothesis generating exercise that they're undergoing.

Yeah. And the fact that their advisor will write in, you know, the margin of their draft frame, the story in. In this way is due to the way publication and getting your paper accepted by journals is incentivized as well.

Right. Ex. And so there is also this cultural element of scientists need to be right. I mean, everybody needs to be right. Am I right?

Definitely.

You know, there's the, oh, well, I set out to support this hypothesis. And look, my data supports that hypothesis, but in a lot of cases, people are going back and changing their hypotheses to fit the data just because it fits the model of publication better, which I think is kind of silly, really. We're not necessarily learning more by doing that and maybe just sort of accepting that, you know, this was a purely exploratory work for sure.

And I love that you framed it in terms of. We've been ultimately shoehorned by the hypothetical deductive model or scientific method itself. By trying to fit everything that we do into these constraints, we're essentially telling. We're not telling the entire truth of our process and the story. And I think this is particularly applicable to data science because as you've said, data science, machine learning, these are essentially very pattern based processes and methodologies as opposed to the hypothetical deductive model. And I think that's probably why there's a lot of pushback from the scientific community on data science as a whole, because it doesn't conform to what the scientific community has been doing due to scarcity of data. Fourth century.

Yeah, exactly. I think that, you know, scientists are creatures of habit and they. I'm gonna go there. Science, let's do it.

Scientists, I'm ready.

Are elitist. And so they think that the way that they were taught and the way that they are approaching problems and that they have learned to succeed is the only way to be successful and the only way to approach things. And I think that that is a symptom of the lack of diversity in science, to be perfectly honest. When you have one way of approaching a problem that works for, you know, the group of people who are in power, they reinforce this way of thinking. And there's a lot of benefit to approaching problems from a diversity of perspectives. And I think that data science is a literal diversifying of how we're thinking about scientific problems.

Absolutely. We'll jump right back into our interview with Kristi Balli after a short segment. I'm back here for another installment of Insights from Computational Education with Neil Brown, who works as a research fellow in computing education at King's College London. Hi, Neil.

Hi, Hugo.

So, Neil, learning programming is all about being a novice and becoming an expert. How are novices different to experts?

So a lot of people think of novices as sort of mini experts, that they operate the same way as experts do, except that experts are sort of faster and more accurate. But what the research shows us is actually that novices and experts, they don't operate in the same way.

Can you talk us through what that means?

So let's take maths as an example. If you've got the equation x squared plus 1 equals 82. Now if you're practiced in algebra and you see that sort of written down, it's maybe a bit tricky while we're reading it out, but if you see it written down, you just take that equation, you look at x squared plus 1 equals 82 and you just say x is 9. You barely even sort of think about it. You just kind of sort of recognize what the numbers are going to be and you solve it because you're an expert. But if you Give that to a school child. Then they're going to sort of, you know, break it down into the basic steps. They're going to try and arrange things so they've got the X's on one side, so they'll sort, subtract one, they'll square root each side and get to the answer that way. So experts tend to solve basic examples very quickly by just sort of pattern recognition. They look and they know the answer, but novices don't have that yet, so they have to do the basic steps. So they really have to do the problem a different way.

So that's a great example from algebra. I'm wondering how this translates into the programming education.

Yeah, and so the same sort of principle applies there. So if you've got, say, a C programmer and you ask them to calculate the average of an array, they pretty much instantly know what that solution looks like and they're just typing in the code from memory, but they don't really have to think about it or plan it. But for a novice programmer, again, they have to break it down into smaller steps so they think, okay, there's an array, I'm going to need a loop, then I'll need to use the array length as sort of the thing I'm looping towards and maybe I'll need some kind of accumulator variable. So they have to really break it down and think through each little steps that they're going to build up, whereas experts just instantly know the answer.

So I guess this must cause problems when experts teach novices.

Yeah, it definitely can. And it's a phenomenon that's known as the expert blind spot. So experts can get quite frustrated that novices just can't see the obvious answer like they do. And they tend to be quite dismissive of all the small steps that novices need, like planning code and breaking it down to steps before actually sort of writing it. But novices do need different ways to grapple with the problem before they can actually solve it. There's a sort of parallel to reading here. As expert readers, we can just read whole sentences without any effort. But if you're just starting to read, you know, when you're a young child, you sound out individual word parts to kind of get grips of letters and you practice that for a while and then you sort of build up into longer and longer sentences and that's how you become an expert. So they don't become experts just by directly copying the experts from the start.

So novices need to think differently from experts. On top of this when experts are teaching novices, the expert really needs to appreciate where the novice is at and how they think about the challenges at hand. Thanks, Neil. Until next time. After that interlude, it's time to jump back into our chat with Christy. So can you speak a bit more to the tension between traditional research methods and approaches and those that are emerging now, perhaps with particular attention to ecology?

Okay, so ecology, I'll give you a little bit of history on ecology, because the thing is, ecology is a really new science. So the ESA, the Ecological Society of America, was founded in 1915, and the British Ecological Society, it was founded in 1913. And so, and it purports itself as the oldest ecological society in the world. And so each of them started publishing their journals in 19. So Journal of Ecology from the British ecological Society was 1913, Journal of Ecology was 1920. And then, so this is when ecology was sort of becoming a thing. It was becoming its own science. And just to give you an example, there's no Nobel Prize in ecology because it really wasn't perceived as a science when the Nobel was being formed as an award for great science. So this is a new thing. And so almost all of the culture of ecology has evolved in the 20th century. There's also a little bit of marginalization of ecology from the perspective of the hard sciences during that time, as late as the 1960s, there's this famous quote attributed to Ernest Rutherford that all science is either physics or stamp collection. And I think that kind of gives ecology this inferiority complex. It's like, no, we're not stamp collectors, we're a real science. We're studying patterns and processes. But it became this cultural thing that to prove itself, we had to be quantitative to a fault. And so we embraced statistics, we embraced theory, we wanted to apply an equation to everything. And so in the time after the Second World War, so ecology kind of slowed down a bit during the Great Depression and the wars. And then we see this huge expansion of the world university systems after the Second World War. Suddenly everything, there's this massive financial investment in university systems. And ecology is a science and it has a place. And so this is the cultural element around when ecology was becoming a recognized science. And at the same time, the hypothetical deductive model was the predominant method for scientific inquiry and Fisherian statistics. So the null hypothesis significance test was the sort of canon of the time for statistics. And so we get this sort of conflation of hypothetical deductive model and hypothesis testing statistics. So the concept of the statistical hypothesis where you reject a null to support your statistical hypothesis. And your statistical hypothesis would be that there are no differences between two groups, for instance, then that's when you accept the null. If there are differences between the two groups, you'd reject the null. And so this idea of that as a hypothesis test and the hypothetical deductive model, I think got really linked in the idea of, in the minds of a lot of ecologists.

And this is something that has stuck around even until now and had, you know, a huge impact on 20th century science. Perhaps you know, these two. Well, the hypothetical deductive model and Fisherian statistics combined are two of the things that have led to the reproducibility crisis we're in now, right?

Yeah, absolutely. And so you can't publish a paper very easily. And if you have no results, that is no statistically significant results, which it's become this cultural thing of, well, a grad student throws away data where they don't see the pattern that is showing up at the level of statistical significance of 0.05, which is the sort of canon, but the information is still there and there's probably other ways to detect the patterns. It's almost funny at this point. So I was recently reviewing a paper where they had a statistical test between every two numbers they presented to the point where it was meaningful. It's like, we have a significantly different number of apples and oranges. And it's like, well, why, why are we comparing them? You're just counting apples and oranges. Like, why does it matter if they're statistically significantly different? And so it's kind of frustrating. It's like this is, this attempt, it's almost a smoke and mirrors attempt to make the science look more quantitative than it is.

And it almost sounds like a parody in some ways.

Right, yeah, yeah. I was going on to a friend about. It's like, well, do we have a statistically significant different number of stairs in our houses or do we just have a different number of stairs in our houses?

Yeah. So what this really means, kind of as we were hinting at earlier, is that culturally, and I think taking this cultural perspective is actually very telling because it makes it directly clear why data science and people interested in exploratory pattern driven research are marginalized.

Right? Absolutely. I've had where is your hypothesis? Written in the columns of more than one of my manuscripts. It's really kind of telling that, you know, we have this really rigid view how we can interpret data. And I worry that scientists will get left behind because other organizations, other agencies, other aspects of society are Fully embracing data science. So for instance, we, we see that data science is highly effective at figuring out people on the Internet. So citizens on the Internet, their, their political affiliations, their shopping preferences, everything. So data science is what led to Facebook advertising to me, a sofa that I really like. So this started happening not long after I got my assistant professor job. So Facebook must know that my income went up slightly and I bought a house and suddenly it was just ads for this sofa that I really, really liked again and again and again. And so Facebook knows something about me that knows how to target advertising. So I am, I would say, a complex system. And so the fact that like, they don't necessarily need a hypothesis to say, huh, Cristy will probably like this sofa. They just know that people like me probably will like that sofa. And if they show it to me again, again and again and again, they might wear me down and I'll buy it.

Exactly. And they've identified that pattern using a scientific model which is not necessarily mechanistic. I mean, they don't know why or anything along those lines. Right. It doesn't explain kind of the reasoning which science, you know, historically has been very interested in, but it can let them know that this pattern exists in order to target you.

Yeah, exactly. And so why couldn't scientists apply this to systems where we need to know an action, but we don't necessarily need to know a mechanism? And so a lot of conservation biology is like that. We need an answer before we can fully understand the. So if we're planting trees, we need to know what is the best practice for planting those trees that's going to make them survive in a depleted landscape. We need to know the patterns and be able to really capitalize on the information that we have in a lot of restored systems. You know, if there's an algal bloom in Lake Erie, this is something that is constantly on our mind here in the Cleveland area. We need to know what to do to act rather than necessarily the mechanism. Now the mechanism is excellent and important too, but if we need to keep the tap water clean, we need to act right away. And so we can do that with pattern driven reasoning.

And I think the statement essentially is that all of these pattern based techniques are part of science. Science isn't necessarily the exclusive realm of mechanistic modeling.

Exactly. Yeah. And so it's all science, it's all ways of knowing, and all ways of knowing how to better our world and how to push the human condition forward, if you will. I tend to take a pretty applied view of science in that. You know, I don't think science is necessarily just an intellectual pursuit. I think it is a social and political and human pursuit.

Yeah, absolutely. And I think that's part of the reason that you've identified or you're interested in a lot of what happens culturally in terms of the methods that we all use, that allows you then to think about the human impact. And I know you're a serious proponent of open science and the effects that that can have on this type of, you know, bettering the world. And we'll get to that in a second. But before that, I'd like to know, kind of pivot back to ecology briefly and hear your thoughts on what the biggest challenges in ecology today that data science can help to solve.

Oh, that, that is a hard one because there's, there's just so much going on. I think that climate change is a huge, a huge, multifaceted, faceted problem. So climate change is not just global warming, and that, that's the thing. So understanding how it's going to play out in different ecosystems is just this incredible multilayered problem that I think that we have to essentially layer lots and lots of data sources to understand how patterns play out in very different ecosystems. Because you can't just build a model and say, turn up the heat and get a reasonable prediction. There's so many layers. So humidity is affected, the soil quality plays into how organisms respond to heat. So there's all these different layers and every landscape is different. And so I think that data science is really crucial to preparing for the impacts of climate change.

Yeah, and I think one of the things that you're speaking to there is just the amount of data we have and the techniques, in terms of actually scaling our techniques to deal with the amount of data and also kind of the heterogeneity in the data that we have data sources of incredibly different types and figuring out how to join and combine them in sensible ways.

Yeah, absolutely. So, for instance, I have colleagues who are working on monarch butterfly populations. And so you might have a survey of monarch larvae that's done in Illinois, and you might have information on monarch migration at Point Pele in Ontario. And then you could have remote sensing data for sensing how prairie patches are doing where they have their milkweed hosts. And so you can the remote sensing data available at their stopover sites in Texas, and then information coming out of Mexico. And so you get this incredible multilayered, multidimensional data set that doesn't necessarily easily mix together. And so we need Advanced techniques that bring it together.

Yeah. And what about the idea of scaling results in the literature and aggregating results in the literature? Because I'm sure if I, you know, I'm not an ecologist, but if I went and tried to find out a bunch about ecology, you know, there'd be all types of different results from different labs, from different people in each lab. How to actually using data science to get a sense of the story as a whole.

This is actually something that I'm planning on working on capitalizing on some long term data. I've got a grant in review about this right now, but the classic ecological study is a three year study. I actually did a search and found that most commonly ecology is done on about a three year time scale. There's two year studies, there's four year studies. But the reason that the unit of time is that's about as long as a PhD student can get in the field and get meaningful data. You know, you have your three to seven year PhD programs and you know, the longer ones are usually because there was a disaster of a field season one of the years. And so like the best PhD programs tend to have about three years of data associated with them. But when you think about ecological processes, three years is really just a snapshot. So what happens when it say for instance in southwest Michigan, the fireflies tend to follow a six year population cycle. So what happens if the student was studying it for the three years that they were going up or the three years that they're going down? They'd have really dramatically different conclusions. And so what my plan is, is to take long term data sets and reanalyze them as if they were three year studies and see if we can find out anything about these three year studies, any sort of characteristics about these three year studies that would tell us how often these three year studies are wrong and how often they are bucking the trend, the longer trend in the system.

We'll jump right back into our interview with Christy Bally after a short segment. We've got another great segment that explores ways to get involved in the data for good movement. Peter Bull, co founder at Driven Data, is with us again to guide the way. Peter, what's the topic this time?

Oh Hugo, I didn't see you there. I was too busy harnessing the power of AI to push forward social causes we all care about. But wait, did I hear you say you're interested in sharing with the audience how to get involved?

I sure did, Peter. You did hear that correctly. We've talked about volunteer contributions for people who have time here or there to donate, but I'm interested in what a professional data scientist can do if they find themselves with more time on their hands.

There are really great organizations for this. One that is really near and dear to our hearts is Datakind. They're one of the longest running Data for Good initiatives and they have chapters in New York, San Francisco, the United Kingdom, India and Singapore. Datakind coordinates weekend log data dives and also longer team based data core projects. For example, they've helped improve financial inclusion in Senegal and built models to understand human rights abuses for Amnesty International.

Amazing. I'm constantly inspired by the data scientists that are solving challenging problems at work and then are able to turn around and do the same for worthy organizations in their free time. Are there other groups folks could volunteer with?

Definitely. Another great one is called Delta Analytics. They're based in San Francisco but have worked on cool projects around the world. They teach a machine learning course in Kenya that helps build local capacity for data science work. They also run data Fellows projects and have had some amazing successes including building algorithms that help to identify logging in the Amazon rainforest by triangulating sounds recorded on old cell phones that were rigged to work on solar power and strapped to trees in Brazil.

Wow, that is impressive to see how interesting and unexpected a lot of these projects are.

I know it's really incredible and that's one of the reasons I love these segments. We get the chance to show that there are so many interesting problems to be solved with data outside of traditional industries. That said, I encourage everyone to talk to their employers about how their own corporate social responsibility and data philanthropy strategies could make a difference from volunteering with local groups to providing novel data. There's a lot of change that can be driven by employees who want to make sure they work at a company that shares their values.

Thanks Peter, I'm inspired. I'll definitely link to these ways to get involved in our show. Notes. Time to get straight back into our chat with Kristy. So I want to now move to something we've been circling around a lot, which is open science. And I know that you're a serious proponent of open science. I'm wondering, generally, what does open science mean to you?

So open science is like this really multi layered, multifaceted thing, but I like to distill it into something very simple. It's the movement to put science as much of the scientific process, into the hands of the interested parties as much as possible. And so this is like for instance, a member of the Public might be interested in a particular scientific paper. And if it's not open access, which is a facet of open science, they wouldn't be able to access it unless they had access to a university library. So taxpayer funded science is actually behind a paywall. They'll have to pay $39 to get access to it or some number that similar to that. But there's also the fact that science has been traditionally published as units of the paper rather than the whole process going into the paper. But the paper is just sort of this end product of the science. It's a summary document at the end of the science that says, okay, wrapping this all up, this is what we found. But the thing is, with science, there's so much involved in the process that we haven't traditionally been able to publish it with the paper. So our raw data sets weren't included. We now use analysis code to do a lot of our analysis in more quantitative sciences. And so we can make those available too now that we have web technology. And so there's all of these layers that you can have that just essentially make it more easy for other scientists and other people to replicate science, more easy for them to understand science, and more easy for them to access aspects of the science through its process. And so it's got lots of layers. So the whole openness and inclusion aspect of things, where we essentially say, well, traditionally science has been a rich, white, European descent man's game and we want to bring more people in and we have to specifically address barriers to how people of the world access science. And then there's through policy essentially making it so that it's. The rules state that we have to make our publicly funded science available. Then there's a huge aspect associated with technology. So without the web, open science would, well, it would be, it would be a lot slower. I wouldn't say it wouldn't be possible, but it would certainly be a lot slower. Through the web we can essentially access information of almost any kind from anywhere. And so there's a lot of technology development to foster different parts of science through the open framework. And then there's open education, which is making educational materials, making learning materials available to people all over the world. And so that's, you know, open textbooks, open course materials and that sort of thing.

Yeah, I think framing it with these four points is, is really interesting. And to me it seems like a no brainer in, in a lot of ways that this is definitely the direction we need to be moving in. You know, and I think people might Be surprised how much pushback there is from within the scientific community towards open science. In a lot of ways. We spoke about kind of the generational nature of, you know, the people in on the National Academy of Sciences maybe didn't need open science to be successful. So there's perhaps pushback there. There's pushback in a lot of other places. I'm just wondering, with respect to pushback and otherwise, what are the biggest challenges facing open science in your mind?

I'd say that it's almost all a cultural thing. Just as you were saying, the very same elements that look at data science with skepticism often regard open science with the same sort of skepticism and for those same root reasons. So we are in a state of late stage capitalism and resources are scarce and we have to compete for them. This is very true in science where we have decreasing rates of grant funding of our work. I'd say that American science is a high stakes, high rewards game. So our job security and our grant money is deeply connected to this idea of, you know, being, being the best scientist, sciencing the hardest. I like to say it as often, you know, and so you actually create a tragedy of the commons. So the idea that open science places its practitioners at a disadvantage in this environment. If you have people who, if you have a system that essentially rewards the most selfish practices, you know, the most self preserving practices in science and then spending time on a more communal practice in science is actually putting you at a disadvantage through opportunity cost and also through alienation of others. So people are distrustful of others that don't share their value system. And so I've had some side eyes where, you know, I say, well, can we publish a preprint on our work? Can we make this available publicly? Like can we aim for an open access journal? Because people don't necessarily trust those practices and you know, people don't want to share their data, they're worried about their data being used by others. They don't want to share the code because they're worried about their code being used by others and not getting properly credited or you know, having their code used against them. Even, you know, someone finds an error in their code and then it's exposed that they're a fraud as a fraud as a scientist rather than a human.

And just quickly people outside academic research may not realize how real a thing scooping is. Right? Yeah, literally people will be productive of their results, but so that other working scientists don't scoop them and publish that result before them.

Yeah, and I'd say That the real risk of scooping does vary with field, but it's not as dramatic as people sort of frame it. But it's part of this oral tradition in science. So I remember grad students taking me aside early on in my career and, you know, telling me to sort of guard my ideas, guard my data, because if someone scooped me, I'd be up a creek.

You know, cost is so high.

Yeah, exactly.

I mean, the risk may not be high, but the cost is high.

Exactly. And so especially for young researchers, having an experiment fail for, you know, a master's student or an early PhD student, that can be career altering or ending. But you know, for an established scientist, it's less risky. However, it's the established scientists that are sort of pushing these ideals down to the less established scientists. So this is where I kind of turn this on its head a little bit, because I think that open science can also be its worst enemy at times. So with every new movement, there are, shall we say, enthusiastic adopters who become really enamored by the purity of the ideology. And so it can in turn become competitive even within open science. To be the most open, essentially, it's like a purity test. And so they get very focused on being the most open ever, that they alienate people who are just trying to learn to adopt open practices, because essentially someone who takes one step, they're essentially telling them you're not open enough, or people who feel that it would be too much a risk to them, they just don't listen to the concerns. They're not necessarily focused, helping the people that they're trying to help. So, for example, I see a lot of people pushing technological solutions to problems that people outside the open science community don't see as problems. And so one of the common elements of the people in the open science community is they tend to be very pro technology. And well, especially in biology, you have people who are a little bit more technology adverse. And so when you propose a technological solution to a problem that people don't perceive as a problem, you get pushback of. You are trying to push me to do something I'm uncomfortable with. You're trying to waste my time. And so this is something that the open science community kind of needs a little bit of soul searching on to see how we can better initiate new members.

Yes. As you've been talking about all these aspects of open science and data science in academic research, there are so many resonances, I think, for what happens in industry. So I was wondering what similarities you see in data science and Industry and academia, particularly with respect to openness in science, kindness in science and lack thereof.

So I think that both data science and open science are movements that can make things better and can benefit from each other. So, for instance, data science really can't happen without open science because data science so heavily relies on data integration. And where's that data going to come from if not from people being open with their data? But I think we need to be really mindful about how we approach both data science and open science to make them kind, like make them positive humanist pursuits that aren't necessarily used for evil. So I used the example of the sofa earlier on. You know, that's an effective use of data science, but is it a ethical one? Well, you know, that's open to interpretation. So open science is a movement that's designed for accessibility and inclusion, but it's subject to the same cliquishness and exclusionary practices and micro and macro aggressions that greater society is subject to. And so data science has been applied in all sorts of various ways, so we won't go into it too much, but you know, to manipulate people into voting a certain way and buying sofas they don't need. So that's not in the public good and it's not kind. I think the core of both data science and open science is that we need good people and taking leadership roles in both fields and advancing both in humanist, kind, inclusive directions.

Yeah, and the example of the sofa is interesting because it's a lot of, it's due to exposure. I mean, we've said the word sofa so many times in the past 40 minutes that I kind of feel like I need a new one, essentially.

I know, right? This, this has worn me down. I didn't even want to get rid of my old sofa. It's fine. You know, I have young children. I should probably wait until they stop spilling things before I get a new sofa.

Or you could just keep getting new sofas.

Oh no, that is a better long term plan. Thank you. Late stage capitalism.

Thank you very much. So what does the future of data science look like to you?

That's a really hard one. I don't honestly know. I feel like we're going to be able to make better decisions, but I think we're kind of at a crossroads in society about how things are going to be regulated. The whole Facebook trying to become kind has been something that's been of interest to me because they are probably one of the greatest engines of data science known to humanity. And, well, they are trying to change their practices, at least outwardly, or at least they're using very effective methods to try and convince me that they're changing their practices. And so I honestly don't know. I feel like we can advance science dramatically through using data science techniques, but I also feel that it's going to depend on the humans that take up the practices is really what it comes down to.

Agreed. So we haven't talked a lot about technical stuff. We've stepped back quite a bit. And I'm just wondering what one of your favorite data sciencey techniques and or methodologies is.

So my very favorite tool is OpenRefine. So OpenRefine, it came out of Google and it is a tool for cleaning data in ways that like cleaning dirty, messy data in ways that are so powerful and also so accessible. So what you can do is you load your data, your spreadsheet directly into it and it examines the data, it looks for commonalities, it examines it for typos. And so you can essentially find problems with your data that would be sort of needle in a haystack issues that you wouldn't necessarily find out about until down the line, if ever. And so it's one of those tools that I use in teaching a lot because not only does it help you fix the problems, but it helps you teach the students how to what the problems really are. So the example that I use in my class is imagine you had someone who did a survey of bees in a crop and they collected all the specimens and they went back and ID them and then they had their assistant enter what species each specimen was. And so you've got thousands of bees and you're entering them in the spreadsheet and your assistant is going to make errors. But how are you going to find the errors in your spreadsheet to find out how many species you have? OpenRefine allows you to locate things that are likely typos. And it brings it up to you and says, oh, is this an issue? Might these be the same species? And this is something that really you could do this through, you know, more command liney sort of higher order processes before. But this provides this really graphical interface of like, just allows you to see where the problems are. And so it allows you to highlight to students, well, this would be a problem if we were counting the number of species in this sample and we've spelled our European honeybee, Aphis mellifera, several different ways. It would count those several different ways all as different species. And how do we resolve that?

That sounds like A really cool tool. I haven't checked out OpenRefine, but I'll make sure to.

Oh, it's so powerful. I love it.

So, Kristy, my final question is, do you have a final call to action for our listeners out there?

Okay, call to action. Wow, this. So open science and data science are microcosms and they're a product of the society that we're living in. And right now they're not inherently forces of evil or good, but we live in a society where things are not necessarily kind. Right now, our economy is not kind. Late stage capitalism is not kind. Individualism is not kind. But the cultural elements shape how the powerful new approaches are applied. So we need data science leaders and open science leaders. Like we need leaders in every part of society fighting for the common good. We can use information to empower and motivate people and bring about better lives for our fellow humans. And I think that's beautiful. So let's use these tools for good.

I couldn't agree more. Christy, it's been such a pleasure having you on the show.

Oh, thank you so much, Hugo. It's been great chatting with you.

Thanks for joining our conversation with Christy. We saw how data science techniques such as machine learning are finally gaining momentum in all forms of basic science research, with particular attention in this conversation paid to pattern finding methodologies in ecological systems. This represents a paradigm shift away from hypothesis driven scientific research, which stemmed in part from the scarcity of data. To be precise, the hypothetical deductive model of scientific inquiry combined with Fisherian statistics, most famous for null hypothesis significance testing, is not the only way to build bodies of scientific knowledge, although culturally in academic research it has been accepted as so. With the masses of data we're now able to collect, compute and process pattern finding techniques are ripe for the picking, but progress is slow and the change is often generational. But we are seeing it. We also saw the importance and accelerating emergence of open science as represented by four openness and inclusion policy, technology and education. Last but definitively not least, we saw the importance of kindness in science. I'll quote Christie verbatim as she said it so well. Open science and data science are microcosms and they're a product of the society that we're living in right now. They're not inherently forces of evil or good, but we live in a society where things are not necessarily kind. Right now, our economy is not kind. Late stage capitalism is not kind. Individualism is not kind. The cultural elements shape how the powerful new approaches are applied. We need data science leaders and open science leaders like we need leaders in every part of society fighting for the common good. We can use information to empower and motivate people and bring about better lives for our fellow humans. I think that's beautiful. So let's use these tools for good. Also, make sure to check out our next episode a conversation with James D. Long, a misplaced Southern agricultural economist, quant stochastic modeler, and cocktail party host. JD is VP of Risk Management for Renaissance Reinsurance. That doesn't make him Tony Soprano, but it does make him an expert in the applications of data science techniques to the omnipresent worlds of insurance, reinsurance, risk management, and uncertainty. What are the biggest challenges in insurance and reinsurance that data science can impact? How does JD go about building risk representations of every deal? How can thinking in a distributed fashion allow us to think about risk and uncertainty? What is the role of empathy in data science? You'll need to tune in next week to find out. I'm your host Hugo Bown Anderson. You can follow me on Twitter Hugo Bowne and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com Community podcast.

Ra.