#12 Data Science, Nuclear Engineering and the Open Source (with Katy Huff)

In this episode of Data Framed, a Data Camp podcast, I'll be speaking with Katie Huff, an assistant professor in the Department of Nuclear Plasma and Radiological Engineering at the University of Illinois, where she leads the Advanced Reactors and Fuel Cycles Research Group. I'll be chatting with Katie about data science, nuclear engineering, the importance of interdisciplinary data science, and the open source. If you want to know how data science and artificial neural networks are being leveraged to help detect nuclear nefarious nuclear material in urban environments, among many other things, you've come to the right place. I'm Hugo Bown Anderson, a Data Scientist at DataCamp, and this is Data Framed. Welcome to Data Framed, a weekly data Camp podcast exploring what data science looks like on the ground for working data scientists and what problems it can solve. I'm your host, Hugo Bown Anderson. You can follow me on Twitter Hugo Bownt and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com community podcast. Hi Katie, and welcome to Data Framed.

Hi. It's great to be here, it's great.

To have you here, and I'm really excited to be talking about data science and nuclear engineering today. But first I'd like to find out a bit about you. What are you known for in the data science community?

I'm not sure. It's probably this O'Reilly book that I wrote with Anthony Scopat's Effective Computation in Physics. Or it could be my early leadership in the Software Carpentry Foundation. I've generally been very involved in the scipy community and in sort of larger discussions around software reproducibility and best practices in scientific computing. If people don't know me from that, they might know me, because last year I had the really amazing opportunity to give some keynotes at both scipy and pycon. So people who didn't know me before that may have seen me give those talks.

And how did you get into data science?

Originally, When I was 16, my first job was as a summer student at Los Alamos National Lab, working at the accelerator. And back then in 2003, whenever we did some data science, we called it analysis. And so as a part of that job, I learned Perl. And that meant that when I got back to undergraduate, they want, you know, I ended up in a research group where I learned C and Python. And then when I went to graduate school, I had these skills already. So I ended up in a really great computational nuclear engineering research group with Paul Wilson at the University of Wisconsin. And, you know, at that time, I then was able to take develop large software packages like the Cyclist tool for fuel cycle analysis and some others. And I got really involved. My advisor encouraged us to have this group of students who were interested in software in general and manipulating data. And that was called the hacker within. We were supposed to find the hacker within ourselves when we ran into a problem.

Did you find the hacker within?

Yeah. So I was a co founder with Milad Fat Nejad and it was really wonderful, I have to say. It was a really great group of people who talked about all the coolest things that were going on in computers. We learned get together and we talked about text editors. And this was circa 2011, 2010, so it was pretty early on. And we helped to jumpstart what software carpentry now is by working with Greg Wilson from afar. It was really wonderful.

Great. And then you went to Berkeley, is that right?

That's right. So after graduate school I went to the University of California, Berkeley's Nuclear Engineering department. And while I was there, the Berkeley Institute for Data Science had just gotten funding from Morin Sloan. And so I applied and I became one of the first cohort of postdoctoral fellows in that institute. Mostly, I would say, because of my experience with the sci fi community and the SCIPY snack. At that point I had contributed to a number of packages both within the nuclear community and within SciPy. And I contributed a lot just in that community. So it was a, it was a good fit for me.

And where are you now?

Oh, I'm a. I'm a second year assistant professor at the University of Illinois in Urbana Champaign in the nuclear plasma and radiological engineering department.

Awesome. And that's what we're here to talk about today, among other things. But that's been a great whirlwind introduction to what you do and your background. And you hit upon a number of touch points which I hope to return to. You spoke about the importance of Git version control software development for research scientists and the packages that you've developed, which is becoming increasingly important in academic research. Touch upon your educational outreach and also the importance of just hacking it out, which I think is definitely a skill that we all need to continue building. But as you said, you're a physicist by training and now you're working at the intersection of data science and nuclear engineering. And I'd love to hear from you about what, what type of questions you work on in nuclear engineering and how they relate to data science as a, as a whole, as a discipline.

So my research group is called Advanced Reactors and Fuel Cycles Right. And basically that, that looks into various ways we could have safer or more sustainable nuclear reactors and maybe treat the fuel differently on the back end, have less waste, have different waste, stuff like that. And the kinds of projects that I'm working on vary. Almost all of them are computational, but a few of them do touch on data science a little more formally. I'd say that the bulk of my research group is doing what you would call sort of traditional hpc, like traditional high performance computing. We do large scale multiphysics modeling and simulation of these advanced reactors. And I wouldn't necessarily call that data science, but a lot of the pieces that go into it, I think people recognize a little more as data science. We work on the Blue Waters machine, which is like the fastest computer on a university campus. It's a really wonderful resource and we get a lot of really high performance simulations out of that machine and then we have to analyze them. And that's a real data problem. Some other stuff that we do is I am lucky to have inherited a wonderful student from a colleague who left academia to go to GitHub to be a data scientist. But that student is working on a project I'm really excited about that's related to integrating the signals from networked detectors for radiation. So low resolution detectors in a network can be combined to give a stronger signal. And the project he's sort of thinking about is he's, he's training artificial neural networks to improve the way that those detector signals are interpreted so that we can localize and ideally identify the isotopes in a nefarious source somewhere in an environment. So if you have some nefarious material, some nuclear material that's not supposed to be in an urban, urban environment, in an urban environment, then you should be able to find it even with just a small number of low resolution detectors. And his algorithms are seeking to make that possible for search and recovery of those kinds of things.

That's, that's really interesting. And are there other applications in terms of, you know, finding radiation? Not necessarily in nefarious situations, but other, other situations we'd want to detect radiation.

Sure. I would say that the majority of the ones that the Department of Defense are interested in are these kind of nefarious ones. But you can maybe imagine a situation where a medical isotope source has been lost. This happens a lot in other nations. We have pretty good regulations on these here. But actually one of the biggest sources of radiation in our lives is medical radiation, and that comes from large cadmium sources and stuff like that. That power our x Ray machines and whatnot in hospitals. Those sometimes are disposed of poorly. So you could imagine using this for that purpose to find it if it was lost in a trash dump somewhere, which happens more than you would think, I'm sure.

And you mentioned the Department of Defense. Do you work with the Department of Defense or there's a funding source or something along those lines?

I typically work with the Department of Energy. So my interests are primarily on the energy side. And even this work is related to the Department of Energy's Office of Nuclear Non Proliferation Security. It's actually the NNSA, which is within DoD. And so ultimately my funding doesn't come from DoD, but some of the, some of the applications are DoD related.

This actually raises another question in my mind. A big challenge for data scientists and technically minded researchers everywhere and all through time is having to communicate technical ideas to non technical stakeholders. So do you have this type of challenge when talking to people in the Department of Energy, for example?

So the Department of Energy is very nuclear focused. Interestingly it's called the Department of Energy, but an enormous amount of its funding is actually dedicated to nuclear energy because it started out as the Atomic Energy Commission as an institution. So from a history perspective, like actually the DOE is quite conversant. But in terms of communicating science to the public and like making my, you know, papers readable and public, you know, press releases interesting, and even just communicating about fear around nuclear energy and nuclear engineering, I think that is a core problem within nuclear engineering. And generally speaking, I think that nuclear engineering is totally crippled by a lack of focus on the importance of the public and the importance of discussing the science at the place that the public wants to discuss it. You know, you have to meet people where they are in those conversations. And I think we've really struggled as an industry and as a community to communicate that there's really nothing to be afraid of.

And is part of that the fact that in academic research anyway, these types of outreach efforts aren't incentivized in the way that we would. We're discussing.

Absolutely. Time is finite. My time feels more finite every day. And generally speaking, you do the things that you're going to get rewarded for over time, and that's research and teaching services, this tiny sliver of it. And no amount of outreach is going to get anyone tenure. So. Yeah, I think that's a really good point.

The other thing that came to mind when you were talking about communicating to other types of stakeholders or people you may want to get involved was your talk at PyCon which, which you mentioned. Which. One of the, one of the wonderful, one of the many wonderful things about, about your talk was that you talked about the scientific research process and nuclear engineering to the Python community at large. A lot of software developers, a lot of programmers, and convinced a lot of people and did your best to convince a lot of people to get on board, right?

Yeah. And I have to say the Pycon community was so ready for it and welcoming and enthusiastic. I got a lot of comments afterwards, you know, people saying that they also felt like nuclear is a really obvious choice for, you know, reducing carbon in the world and that they were glad to hear someone say it really confidently and hear a few of the, like, key facts behind that. Guido was enthusiastic. He had positive feelings about the whole gist of that conversation as well, which brought me joy.

And you also, something you did was you framed the open source community as being one of the communities that employs the scientific method best in society. So maybe you could speak to that briefly.

Absolutely, absolutely. So I think the open source community is transparent. It tries to be, you know, software in general tries to be reproducible and automated and that really breeds objectivity in science. We really could use a lot more transparency and openness, a lot more, you know, kicking the tires of everyone's sort of experimental procedures or research software, you know, but some of the incentives are a little wrong in science to make the scientific approach of like sort of openness and peer review really shine. There's a lot of like selfishness and fear that someone's gonna scoop you. And so people are afraid to put what they're working on out there until it's like totally done. And at that point no one's gonna check it out or try it. And like, it's too late even to say, oh, hey, actually you kind of got some pieces of this wrong because like it's already published. You have to like publish a retraction and blah, blah, blah. So we need to be a lot more like the open source community in SC Pull requests and everyone should be looking at everyone else's code and trying to use everyone else's code and building on each other's work rather than reinventing the wheel all the time.

Yeah, and I'd love to see some sort of, I've got a very vague vision of some sort of version controlled science where you see everyone's negative results in the, in the commit history and all the models that didn't work before they. The model they published.

Right, for sure. I have to say, like negative results is really like the Achilles heel of our current system. Knowing what didn't work is so much more valuable than knowing what did work. Knowing what did work is one thing you don't need to try. But like knowing what didn't work, that's tons of things that you now know you don't need to try.

So back to the type of things you're interested in. I know you also have an interest in machine learning.

Yeah, I do. So I have an undergraduate working on something very mathy. We're trying to accelerate Monte Carlo neutron transportation with some machine learning algorithms, in particular unsupervised learning, which is going to be a little tricky. But her name is Xin Wen and she's funded through the national center for Supercomputing Applications SPIN intern program. And she's really a CS student, but I have her improving one of our core algorithms in nuclear engineering, Monte Carlo neutron transport. So Monte Carlo was invented to represent this neutron diffusion problem. How many neutrons are in the reactor over time as a function of all the fissions that are happening and whatnot. And we would love to make Monte Carlo much faster. It's very accurate, but it's not particularly fast because you have to run all these histories, you know, you have to throw the dice many, many times.

And essentially this is a simulation process, right?

That's right. So we're simulating neutron interactions in a multiplying material. So like a reactor.

Cool.

And we use this algorithm all the time in nuclear engineering. The only way we know to make it faster is to solve the equation deterministically beforehand, badly, and then employ that, that solution to help the Monte Carlo program choose the right probabilities and reweight statistics a little better. But instead of doing that, we could maybe bootstrap with more machine learning. So I'm going to try, I'm going to try to see if that speeds things up. But basically it's really algorithmic and like once you get deep into it, it's really just an algorithm on top of an Algori trying to make that first algorithm learn from itself.

Great. And it's interesting that you mentioned simulation again because you mentioned that previously in terms of the large scale multiphysics modeling and simulation on the HP using hpc. And I think it's been really interesting in the past few years seeing simulation become more and more a part of what data science is in terms of getting to know the system you think you're seeing, right?

Absolutely. And I think, you know, even statistics itself, you know, it can be really easy to get the right statistics out. If you do it by simulation rather than by some equation that, you know, your statistics professor wrote down five years ago, simulation is really accurate. It's, you know, we have a lot of CPU hours to blow, so we can, we can burn them these days in a way that we didn't have that capability before. So, you know, leaning on the computers to do some of that math and to, to really, like, do our hard thinking for us, I think is, is a really good thing.

Absolutely. We may as well utilize these, these big computational engines we have, because I probably say this on, on, on too many episodes of, of Data Frame, but Daniel Bernoulli, Pascal Fermat, they used to sit around literally flipping coins. They flipped, you know, Bernoulli flips 7,000 coins in a row or something to get, to check out his binomial distribution and that type of stuff. So if he could do it like that, we should definitely be using, using our computational power to do it.

Absolutely. And in fields like mine where you can't just go out and build a molten salt reactor tomorrow, it's expensive, it's hard. You have to have a number of government institutions involved. It's much easier to simulate it and get that, get the design really, you know, nailed down before you suggest building anything.

We'll jump right back into our interview with Katie after a short segment. Now we have a segment called Data Science Best Practices with Justin Boyce joining us once again from Caltech, where he teaches in the division of Biology and Biological Engineering. Justin is also a data camp instructor. What have you got for us today, Justin?

Hey, Hugo, Great to be back. In the last segment, I gave a simple plot all your data. Today I'm going to talk about how swarm plots and related jitter plots are a great way to do this, essentially as a drop in replacement for bar graphs. First, I want to recall the moment when the plot all your data mantra was first impressioned upon me. It comes from someone you and I both know and love, Joe Howard from.

Yale University, my postdoc advisor.

Yeah, and I wrote one of my favorite papers with Joe. I remember Joe always stressing to plot all your data. Well, that's the main point. There were usually expletives in there too.

I remember it well.

Okay, now it's hard to talk about visualization in an audio podcast, so there's a link to some sample figures and code used to generate them in the show notes.

There sure is.

Now, as is often the case, the ideas I'm putting forward are best illustrated by Example, One of my favorite data sets of all time comes from Peter and Rosemary Grant. They've gone to the Galapagos island of Daphne Major every year for decades to study the famous Darwin's finches. Now, there are two main species called Fortis and Scandens. In 2012, for example, they measured the length of the beak of every finch on the island. That amounted to 120 Fortis birds and 126 Scandin's birds. Now, let's say I want to make a plot of the beak lengths for these two species. One reasonable alternative to a bar graph for displaying this data set is a box and whisker plot. But a better option is a swarm plot, also called a bee swarm plot.

Tell us about these.

Okay. Imagine dividing the plot into two areas. The left area for Fortis beak lengths and the right for scandids. The Y value or position along the vertical axis of each dot in the left region is the beak length of a Fortis bird. And the Y value of each dot in the right region is that of a Scandens bird. The exact X coordinates of the dots are immaterial, provided they fall in the appropriate region. So every data point is plotted. You can see how the measurements are distributed and make more thorough and careful comparisons between the two species.

So how do you make sure the data points don't overlap in a swarm plot? Or I guess more specifically, how do you set those immaterial X values?

Well, you usually don't yourself. There are algorithms for making swarm plots that do this for you and ensure that the plotted points do not overlap. Lots of statistical plotting packages like ggplot2 and Seaborn have this capability.

Well, you can make a swarm plot with 120 data points. Sure. But what if you had 10,020 data points? Wouldn't that be too crowded?

Yeah, you really couldn't make a swarm plot with a large data set. When this happens, a jitter plot is a better option. Here you still plot each point in areas reserved for fortis and for scandens. Again, the Y values are the measured beak lengths. The difference is that the X values are randomly chosen within the respective region of the plot. That is the points are jittered, so you do get overlap, but you are still plotting all points.

If the data are overlapping, how do you make sure you can still see them all or at least determine the relative density of points?

There are a couple options. The simplest is to specify transparency or alpha in the data points. Many overlapping points appear darker, so you get a sense of density. This usually involves a little trial and error to get the best transparency value for your plot. Another option is to use more sophisticated data shading techniques using tools like Data Shader, which is part of the Python data Science ecosystem.

Thanks for the tips, Justin.

My pleasure. Next time I'll talk about my preferred plotting method, ecdfs. Can't wait to come back.

I look forward to it. After that interlude, it's time to jump back into our chat with Katie Huff. I think something that I've noticed throughout this conversation is your research really touches upon a lot of different things. I mean, you know, the detection algorithms speak to something incredibly concrete. Then you're working in speeding up Monte Carlo with machine learning, so using data science techniques to do some pretty heavy algorithmic optimization. But you're also doing a lot of modeling, simulation, and also developing software tools. So you're really covering a lot of different bases there. The development of software tools is something I'm very interested in. And I'm wondering, in your research in your department and broader academic research at large, how important is it for people to be software literate and even be development literate?

I think being software literate and development literate makes it possible for me to approach all of these problems. Without that literacy, I can't build the tools that are going to make all of my problems more efficient and automated. I really think that a lot of people in academia don't recognize the importance of that yet, but many are starting to. You see some really impressive things coming out of academic computing institutions these days, and that really comes from academics who can cross that line between software and their application research. Because nothing will hobble you more than buggy code that you don't understand very well that some graduate student wrote three years ago that the advisor never touched. It is so important for just like an advisor in an experimental group might understand the experimental apparatus, but, you know, inside and out. If you're a computationalist, you know, as an advisor, as a, you know, PI, as the leader of some research project, you really have to know that software inside and out in order for students to understand it, in order for continuity of your research group, but also just in order for you to use it and then explore. The end goal for me is never the software. It is what problems can I solve with this software?

It's a tool.

Yeah, yeah, great. And I think just, you know, increasing computational literacy and encouraging people to think computationally and in terms of their data is incredibly important. So, as you know, my previous incarnation of myself worked in biophysics And I worked with a lot of fantastic grad students and postdocs who would pipette for four years, you know, and, you know, they could have done it for two years and realized that they had enough data to make conclusions and get results out at, at that point. But it just wasn't necessarily incentivized and it wasn't actually part of the educational system for them.

Right.

Because they weren't taught these things necessarily. They were taught statistics classes a lot of the time, which taught them, you know, the central limit theorem with a proof by calculus, as as opposed to looking at data and seeing it, seeing it emerge. So I think we're going to see a more and more increasing emphasis on this in early education, hopefully. As we discussed, you used to work at Berkeley, at bids, which is renowned for its interdisciplinary approach to data science. What gains are to be made by taking an interdisciplinary approach to data science in your mind?

You know, data science is such a weird term. It has no real domain without any kind of real tangible data of some kind to work with. Data science is just algorithms, which, you know, on their own are a little nebulous. So to do data science, you really have to have some data. And that data has to come from somewhere. Maybe it's names, numbers, particles, DNA, latitude, longitude, who knows? But it has a meaning, some domain, some discipline. And as soon as you add that discipline to the sort of discipline of data science, now it's interdisciplinary. You know, someone who does data science for Google or Amazon or some other Internet machinery is really engaged in the domain of whatever the data is. If they're predicting movie choices for Netflix, they're in the entertainment domain. If they're increasing ad clicks, they're in advertising. The same is true in kind of more sciency parts of data science. People who are doing data science have to have this data. Biologists have data, physicists have data, sociologists, sociologists have data. And then they become interdisciplinary when they start using computing and thinking about the science of computing. I'd say that even beyond that, at bids, you know, they were very interested in and people from completely different scientific disciplines meeting at the algorithms. So when I do nuclear engineering with my nuclear data, I sometimes use sophisticated techniques. And maybe it could be called data science. And those techniques, those are what can translate from me to some astrophysicist or sociologist sitting next to me. So that institute actually was really good at having people from completely different scientific domains meeting at their shared domain of data science, because then we could talk about the algorithms and the techniques and the Software testing and all of that.

I like that a lot. And there's a lot to unpack in what you just said. But something that really stuck out, I think, is you're speaking to the fact that the interdisciplinary approach means that we don't have to continually reinvent the wheel in silos or in different disciplines, so to speak.

Yes, absolutely. You know, sharing is how science is supposed to work. We're supposed to build on each other. And there's so much to keep track of these days. Like you cannot possibly know what's new in biology if you are a physicist. It's just too much to keep track of. But if you can somehow meet at the things that you agree at and the things that you share, then maybe you can keep track of sort of the advancing front of data science.

And are there examples and what type. Are there any techniques that you've used that haven't been developed in nuclear engineering, that have appeared in astronauts, astrophysics or any. Anything along those lines?

Well, I would say that like all of the software development techniques that I use to create the kind of simulation software in my research group, these are things that I learned from the broader SCIPY community, not at all from nuclear engineers, and, well, a little bit from nuclear engineers. I have to give a huge amount of credit to my advisor, who is the coolest person in the universe, Paul Wilson. So he did have a lot of these ideas and inherently. But, you know, I learned so much from software carpentry, from SciPy, about how to put together a really reproducible application, how to put together an appropriate database, how to apportion runtime properly, and how to do unit testing and what kind of data structures are going to work for any of this. I did not learn most of that from nuclear engineers at all. I learned it from these other communities.

And that's the key term that we've actually been circling around is community.

Right.

And in fact, this is one of the reasons, anecdotally at least, that both Python and R have been so embraced by practitioners is because of the communities that work with them and develop them. So two of my favorite conferences are, as I presume yours are PYCON and scipy, because it's so great to go to them and see everyone and talk shop with everyone.

Right? Yeah, absolutely. I look forward to SCIPY every year. It's the greatest place, full of the most excited, wonderful people doing some of the most amazing things in the world.

I think something you're speaking to is finding communities that break across disciplines as well.

Right.

So maybe you can speak to that a bit more.

Yeah. So I think, you know, for me, that all started with the hacker within, which really attracted people not just in nuclear engineering, but across campus, who were maybe the only graduate students in their research group that was doing any kind of computing and developing. That community was great because it was a bunch of graduate students with their own endogenous interest in this topic. But, you know, it can be very hard to formulate a community that's cohesive and sustainable. The hacker within exists in a number of places now and it has had various success, you know, based on the region it's in or the amount of leadership or the amount of enthusiasm, sometimes just the amount of free beer or pizza that was involved. I would say that one of the best communities that's probably really exemplified community development and sustainability for me watching it has been software carpentry and data carpentry, which are now together called the carpentries. Ultimately, those communities which get scientists to teach other scientists how to use computers properly, those communities are really phenomenal because everyone shares this endogenous interest in making sure everyone knows how to do all of these things properly in all of science. I think we all really share that dream.

Yeah. And the other thing about the carpentries that I have found incredible is that it's a community of hundreds and hundreds, probably over a thousand now, volunteers essentially, who love going and teaching and seeing how these skills can help people in their research and in their lives.

Yeah. And it's so rewarding.

It really is. And the other thing to mention, of course, is that the carpentries are having their first conference this year, Right?

Yes, carpentrycon. Are you gonna go?

I actually won't be there. I'll be in Australia at the time. But it seems super exciting. And if any, any of our listeners out there are interested, check out, check out Carpentry Con because it should be an incredible event.

And it'll be in Dublin the like last day of May and the first day of June this year.

So check it out and you'll be there.

Katie, I am hoping to make it, but it kind of depends on some things. We'll see.

Okay, so we've talked about a lot of the ways in which you use data scientific techniques and think about data science in your work. What do you see as the main challenges facing this emerging nebulous discipline of data science?

Yeah, as I mentioned earlier, I think simulation is great because we can really lean on the computers, but we can't lean too much. I think I Hate to get too academic, but it really comes comes down.

To get academic, you're an academic, to.

Really rigorously interpret the results of any kind of model in the context of the model. And it's really tempting with all of these great, user friendly, robust, easy to use algorithms available to us, it's easy to treat them as a black box. Too easy. You know, it's tempting to just throw your favorite algorithm at some hunk of data, call it a training set and start making some predictions. Yeah. So ultimately like it's really, you absolutely need, you need to understand how all of these answers are arrived at in order to trust the answers that you're getting. And we're getting to the point where it's sort of like how many times do you open the hood of your car and understand what's really wrong with your engine? I think these algorithms we need to keep, keep the hood open a little bit.

And how do we do that?

Yeah. So I think especially you can do this sort of in the open source sense with peer review. Ask your friends, how did you select that particular artificial neural network model? How did you select the training set? Should you be doing unsupervised learning? Or is there is this problem maybe more appropriate for a supervised technique? I think asking each other and sort of querying those results, sort of as a community, here we come back to, as a community, I say like use that open source ethos, peer review each other's work and make sure that everyone's keeping themselves accountable as to what modeling decisions they're making.

Yeah, I think that's right. And I think, you know, developing that system of more open communication around what we're doing and why we're doing it is incredibly important. The other thing that I think is really important is don't just start throwing models at your data. I mean, exploratory data analysis is an incredible tool and we have all the tools around it, such as Jupyter notebooks, to really do a lot of robust eda. And John Tookey among many other people, came up with an incredible approach to really exploring your data and thinking about it before starting to throw, you know, X layer neural nets at it.

Exactly.

So as we've discussed, you're super passionate and wonderfully passionate about the need for building software tools in basic science research, particularly when viewed through the lens of reproducibility. So could you speak more to why this is such an important concern?

Yeah. So science, it's the key to human progress. Breakthroughs in science, they're not going to happen without excellent software tools. Excellent software tools. They're only going to happen if they're well tested because no one writes bug free software. Everyone, everyone on the planet is going to write a few bugs into their software. They're going to like, make a little mistake here and there's. And you cannot have that in science. We are seeking truth, and the truth only comes out when you are really rigorously doubtful that it's going to succeed. So rigorous doubt is kind of the philosophy behind the scientific method. And rigorous doubt is what we get to when we really interrogate our software, when we robustly test it, when we keep it distinctly version controlled, when we allow it to undergo constant peer review and pair programming and testing. And all of these things really come into that. So that's why I'm really passionate about it. It's because science is essential for us to progress as a species, for sure.

And how does this relate to the importance of reproducibility?

Yeah, so if you can't reproduce it, then it's not going to help with the breakthroughs. Right. The really excellent software tools that we have to rely on, they need, they need to help us, not hinder us. So you have to be able to reproduce results to believe them, you know, and believing the results is the first step towards actually, you know, using them in the world.

And what does reproducibility look like in practice to you? I mean, I understand the concept of someone somewhere else being able to reproduce your results, but does this involve experimental setups and software and figures, or could you break it down for us how it looks in your mind?

I think for me, you know, reproducibility really happens when I produce a result from some simulation and then someone later takes that simulation, can add to it, can change it, and can get new results with the same kind of tools and data that I used and extend what I have done. Because science is really all about extension, you know, we just need to reach out further every time we do something new. So I need, you know, not just my, my results to be reproducible so that someone can trust them and believe them, but also so that someone can use them, you know, and maybe those simulations eventually do become a physical item that gets built. You know, ideally we will build some much newer reactors. Right now we have built some new reactors. China's building a ton of reactors all over the place. The US has a few reactors under construction, you know, but is there a new horizon where we start building slightly newer ones with, you know, very different. Different behavior? Yes. And only if you can really solidly trust all of the work that I and other, you know, advanced reactor nuclear engineers have done.

Yeah, And I actually want to get slightly technical here with respect to being able to reproduce computational results. Is being able to reproduce what your computer does or what it looks like on another laptop or another computer super important to this? And I actually mean by that, recreating the environment, using virtual environments.

Yeah, so using virtual environments is useful. It's a snapshot and it allows you to not have to go through this process of building and it allows you to really trust that what you're calculating is the same. Now, I would say that there's kind of different notions of reproducibility. There's replicability, there's kind of computational reproducibility, and then there's like a reproducibility where you have some same measurement that's obtained with the same precision by some other team, a different measuring system, a different location, multiple tiles, et cetera. Then there's replicability. I think what you're talking about when you talk about virtual machines is really a computational replicability of the results. And I would say that there's, there's a number of different sort of goals within reproducibility. And I would say that the, you know, for me, computational reproducibility is going to be a little bit more like being able to duplicate the results of a prior study using the same materials that were used by the original person. So you might use the same raw data to build the same analysis files and implement the same statistical analysis and get the same results. And that's a little bit more like really reproducing the measurement in another lab.

So the following question may be slightly boring, but one of the reasons it's boring is because it's actually one of the most important things ever, I think, in terms of someone else being able to reproduce your code, how well written or commented does it have to be.

Very well, even just for you to be able to reproduce your code a few months later? It needs to be really well documented. The documentation needs to be automated. It can't be in comments hiding somewhere. It needs to be something that an automatic documentation system like Sphinx or Doxygen can create the documentation. If you can't compile the documentation into a like readable format, then it's really not going to be very useful because someone's going to have to dig into the right function to find, really needs to be in a compilable fashion, you know, buildable along with your code and.

I love that you say you're writing it for yourself in the future, because in all honesty, most weeks, one of my least favorite people is me a couple of weeks ago. When I'm reading, I'm reading, reading my code.

Absolutely.

We'll jump right back into our interview with Katie after a short segment. It's now time for a segment called Rich, Famous and Popular with Greg Wilson, who wrangles instructor training here at datacamp. Hi, Greg. G', day, Greg. What do you have for us today?

I'd like to talk a bit about using data science to design programming languages. Anyone who's taught math or programming for any length of time knows that some kinds of syntax seem to be easier for newcomers to learn than others. For example, I think Python is easier than Java. And looking at the number of universities that has switched from Java to Python, it seems that most computer science educators feel the same way.

But everyone has stories like that about their favorite language.

That's true, but a few people are trying to be more rigorous. Back in 2011, Andreas Stefik and his colleagues reported a study they had done that compared the accuracy rates of novices writing programs in three languages. The first was Perl, which is what their school was using in its intro course at the time. The second language was Quorum, which he and his team were building with the goal of making it accessible to people with visual disabilities. And the third was a control called Randomo, and its syntax was generated by rolling dice to select character tokens at random.

You're kidding.

Nope. They figured it would serve as a valid baseline for comparison. What they found, to their surprise, was that Perl was just as hard to learn as a language with a randomly designed syntax.

Okay, now I know you're kidding.

Nope. But a lot of people reacted the same way as you. So they did a follow on study with three different measures and six languages. Perl, Java, Python, Ruby, Quorum, and the same random control language. Once again, they found that the curly brace languages like Perl and Java were just as hard for novices to master as a language with a randomly designed syntax. Python and Ruby were significantly easier, and I mean significant in the statistical sense. And Quorum was easier still, in part because they were ab testing each new feature before adding it.

That's fascinating. So what impact has this work had on language design?

Unfortunately, almost none. Language designers seem to think that usability is in the eye of the beholder. For example, in their second study, Steffig and his colleagues found that the three most common words for looping in computer science for while and for each were rated as the three most unintuitive choices by non programmers. Things like this might not make a difference to experienced coders, but testing our assumptions about languages the same way that we check the design of our websites could only make life better for the 99.9% of people who don't fit that description.

So what's your ask? This week?

I'd like to see a similar study done to compare the usability, in particular the usability for novices of Python, R, Matlab and Julia. We tried to get something like this going with Stefik a few years ago, but we never got to critical mass. And if listeners are interested in helping us take another stab at it, I think the results would give us all something to argue over for years to come.

Thanks very much. If anyone in the audience is interested in giving this a try, please get in touch. We'd love to hear from you. Thanks Greg, and looking forward to speaking with you again.

Thanks Hugo.

Time to get straight back into our chat with Katie. So we've discussed a bunch about educational outreach, such as hacker within software, data carpentry, what's happening in academic institutions, and I'm just thinking about the future of data science and scientific research as a discipline, and I'm wondering how important education is to the discipline going forward.

Well, it's essential, I think. I come from a family where our core value has nothing to do with morals or God or anything like that, but education. So the core value of my family is like education is the utmost, most important thing that you can do for yourself and for the world and for the future of data science. Oh my gosh, so important. It's all about algorithms. This is a very sophisticated sort of environment. We're talking about computers, we're talking about data cleaning, we're talking about real algorithms and applied math. There's no way we're going to progress without really solid, fundamental, solid education all across ages. And in particular, if you don't start young with this, it's going to be a problem later on. And so actually my husband's a mathematician and we talk quite a lot about whether or not high school should probably lead towards advanced statistics rather than advanced calculus for most students. Not saying all students. I'm like, I have a physics degree. Calculus is way more important than statistics in my life.

But and it is for physicists and engineers, but arguably in terms of around, you know, for civilians and citizens in a variety of, of careers and professions, data analysis and statistical thinking, so to speak, are Arguably even more important.

Absolutely.

And also so my background's actually in pure math and I do the amount of integrals I computed between year nine and year 12 was unnecessary. We've looked at how important education is to the future of data science. I'm wondering what the future of data science looks like to you in the next two, five, ten years.

Well, it's taking over everything. I think it's great. Ultimately, I think we all hope that computers are going to someday fix all the problems that humans are too human to fix by ourselves, including, you know, hunger and war. But that's probably not going to happen in the next few years. I think what we're going to see in the next few years is everything is going to be really targeted towards application. So advertisements are already targeted towards you. Ubers come straight to your feet. All of that is sort of enabled by data science. But I think even just public services are starting to see data science improve their capabilities. You know, housing departments are starting to see how data science can improve the way that they like distribute equal housing in the public sector. It depends on your municipality, but some cities have really taken this on and I think as our infrastructures change in cities and in the world, you're going to see data science impacting things that are otherwise really slow moving, plodding sorts of institutions. Our governmental institutions are going to have to improve over time as we start implementing these really helpful predictive algorithms. I am hopeful that it's a positive thing and that it starts to touch all of the things in our lives, even though it's very plodding, so slow moving kinds of institutions, which I think is what we need to look out for is like that's where the real change is going to come. It's in the places where our infrastructure will have to change.

Yeah, that's great.

And you've spoken to several potential positive impacts of data science, but we do need to be mindful of the potential negative ones, particularly in terms of thinking about bias in machine learning. Algorithms is the first one that springs to mind, but there are several others in which we do need to be ethically minded.

Right, Absolutely. So I think Cathy Oneills book, Weapons of Math Destruction. Oh my gosh, it's an amazing book. And ultimately I have to say she touches on a million different ways in which this could all go terribly wrong. And I think it all really comes back to knowing what model you're putting out there in the world and being very, very careful about the implications of those assumptions and how the human interpretation of your results needs to be in the context of the assumptions that your model made. Again, we're back to this black box. But I think the more that we can really make sure that what we're doing keeps in mind all of these sort of externalities of our work, then hopefully we can, we can keep some of those things from happening. But, yeah, enormous amounts of destruction could happen due to the state of science.

Yeah.

And you've said something incredible in there, which has put the, and necessarily so put the onus on the, on the data scientist or the researcher, though, that it's the person building the model that needs to take the responsibility over this, not necessarily a legislator or another stakeholder that when building models, we actually need to be thinking about this concretely.

Yeah, I mean, I think of this as a scientist too. You know, should Alfred Nobel feel a little guilty for helping with, like, war machines? Yeah, I think so. I think science in general needs to be aware of its impact on the world. Same with data science. You know, as a data scientist, you have this kind of responsibility of a scientist to communicate your results in a way that the public can understand them in a way that they will be interpreted properly.

Absolutely. There is a danger there, or it's a very complex discussion. Right. Because I think, correct me if I'm wrong, but the Monte Carlo techniques that you use and a lot of people use were developed during the Manhattan Project.

Absolutely. Yeah. So John von Neumann and Stanislaw Ulam together created this algorithm to represent nuclear, like neutron transport, which is the same physics that I'm trying to represent inside a reactor that's built for energy. Right. It's the same physics of a fission reaction that we're trying to demonstrate for them. They were, they were directing it to the first atomic bombs. So you know that, that Monte Carlo algorithm, you know, it was there to help us better understand those weapons.

So we've discussed a lot of different things about your work in nuclear engineering, about data science, different techniques you use. But I'm wondering, what's one of your favorite data science techniques or methodologies?

I'm going to go with regular expressions. Anyone who knows me will not be surprised.

That is awesome. I love reading.

Who the hell knows? I think it's data science because the first thing you have to do with any data set is fix it. And generally speaking, it's also really helpful for, you know, navigating the file system that you live in all day, every day, if you're doing any kind of software development at all. So regular expressions is just this like patterned just for those who maybe are not super familiar. It's like what you use with grep, right? It's the asterisk means any character. The you know, up caret means the beginning of the line and together you can create these patterns that string match across files. And so this is essential for data cleaning. It's really essential for finding your way around the file system. And I use regular expressions more than any other sort of core tool and it's not tied to any like particular software. It's like a language and if you are comfortable in regular expressions, you should be able to use it in all of your favorite tools. There's a Python regex package, etc.

But yeah, I import re all the time.

Yeah, exactly, exactly.

It's so important and I just love that nothing has replaced regular expressions. Like you'd think that there'd be something we've created that we can do this, but no, it's still regular expressions and we still have to learn so much. Like every time I have to do regex stuff I still have to go, okay, this is going to take some time now, let's get in the way. So that's great. Regex are incredibly incredible, important. And of course the other thing about Regex is that, you know, most of the world's data is, is unstructured text, text data. So it's incredibly important for just parsing stuff that we encounter in our jobs.

Absolutely.

What other techniques do you think are super important for aspiring data scientists and research scientists? I mean, you spoke about version control before unit tests, Regex. Along these lines, what are really the essential skills that you'd urge people to get under their belt?

Yeah, I would say data structures, databases. So I think you should really have one kind of database at least that you're comfortable reading data into and reading data out of. I don't care what it is. Pick your favorite. You should have a favorite kind of scripting or development language. I like working in Python, but yeah, you should have this kind of preferred language where you know the ins and outs, the tips and tricks. Generally speaking, I'd say you have to have Git version control if you want to live in a different world, then you can also learn Mercurial and CVS and SVN and a whole fleet of other things. But today's world is really very git focused, so you've got to have git in your life for everything. But yeah, I would say other things that kind of come into play that people don't pay enough attention to are like licensing. It's really important to know what license you're putting on your data or your software and to really think intelligently about it. Beyond that, I would say as long as you're, you've got the scientific stack of libraries that you're comfortable with in your preferred language, then you are going to be flying. So in, in Python, that is the scipy stack, you know, you need to have your numpies and your scipy Dask, you're going to need scikit Learn and scikit Image and all of these beautiful tools are available to you. So start to really dive into those packages and I would say any kind of starting data scientist is going to fly. Pick your favorite language and within that language find the scientific library stack and start understanding it.

Get in the weeds.

Get in the weeds.

Can I just ask quickly, do you have a favorite license to use for your data and code?

Yeah, So I prefer BSD3. So that's the Berkeley Scientific Distribution Cause 3 type license and it basically says use it for anything. Attribute me if you can, don't sue me if it's wrong.

Great. Yeah, that sounds pretty reasonable.

Yeah.

So to wrap up, I'm just wondering if you have a final call to action to all our aspiring data scientists here.

Yeah, I would say open up the hood, look into that black box, get in the weeds a little bit because you have a sort of inherent responsibility ability to understand your algorithms and to communicate them properly. So it's a little bit of work. But all good things come from some little bit of work. And you can, if you, if you aren't interested in looking in your own algorithms, like maybe, you know, go to your friend and say, like, you know, maybe we could review each other's modeling choices. Look at, look at some great projects and see what kind of modeling choices they've made and why they've made them. Some people are amazing and somehow find the time to blog about this kind of thing. Like Jake Vanderplaas, I know you have spoken to and you know, he's really good at this kind of thing where he communicates the like thought process behind what he's doing. And I think you could maybe learn from some of that.

Katie, it's been such a pleasure chatting with you on the show.

Oh, it's been great to be here. Thanks for this conversation, Hugo.

Absolutely. It was so much fun. Thanks for joining our conversation with Katie about data science, nuclear engineering, the importance of interdisciplinary data science and the open source. We saw how data science and artificial neural networks are being leveraged to help detect nefarious nuclear material in urban environments, and how data science and machine learning can help to improve algorithmic efficiency for the nuclear sciences. We also saw the essential need for communication in a field that impacts the public and society so much, as well as the need for increased computational and data literacy in society and open science. Make sure to check out our next episode, a conversation with Mike Tamir, Head of Data Science@Uber ATG Uber self driving, a machine learning specialist and data science faculty at UC Berkeley. Mike and I will be talking about how he's leveraging data science to build fake news detection algorithms, along with his work in building data science workflows for Fortune 500 companies. I'm your host Hugo Bown Anderson. You can follow me on Twitter Hugo Bown and datacamp at datacamp. You can find all our episodes and show notes@datacamp.com Community podcast.

Sam.