Building the world's most popular data science platform

And ultimately we don't need Docker actually. Turns out if you build things in the right way, you can have incredibly portable side by side installable native executables that work just fine. And that's essentially what the CONDA system is about.

Bandwidth for changelog is provided by Fastly. Learn more at fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean Droplets manage Kubernetes, Manage Databases, Spaces Object Storage, Volume Block storage, Advanced networking like Virtual Private Clouds and Cloud firewalls, Developer Tooling, the Robust API and CLI to make sure you can interact with your infrastructure the way you want to. DigitalOcean is designed for developers and built for businesses. Join over 150000 businesses that develop, manage and scale applications with DigitalOcean. Head to dot co changelog to get started with a 100 credit again dot co changelog.

Welcome to Practical AI, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@change dog.com community and follow us on Twitter. We're at PracticalAI FM.

Well, welcome to another episode of Practical AI. This is Daniel Whitenack. I am a data scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a principal AI strategist at Lockheed Martin. How you doing Chris?

I am doing very well. How's it going today Daniel?

It's going pretty good. Yeah. It's got a chance to get outside a bit over the weekend even if it was just in my yard doing doing mowing and yard work and that sort of thing. So good to get away from the screen a little bit. And what about yourself?

I tried to do some yard work myself and for anyone who listened to us a couple of weeks ago had the broken rib and I discovered it is not as healed as I was hoping it was and so I stopped doing yard work that's no good. And took lots of medication and thus I'm here and everything is fine.

Okay, good. Yeah. The other thing I did was actually some people asked me about my AI workstation build in our Slack channel, so that's up and going. I have that pretty much running 247 with some type of model training, but I have a colleague who there's two GPUs on it, so he's running some stuff on the other one. And I tried to set up like I got a new router for my. I intended to like put in this new router at my house and set my like other router and bridge mode. And like I had this really nice plan and you know, as VPN access and all of this stuff and all of that completely failed. So of course, you know, more network things in the future.

But yeah, never works the first time.

Exactly, exactly. Well, speaking of very practical things and also, you know, setting up environments and all of those things, we're really excited today because Chris, we've mentioned so many times on the podcast, we've mentioned Anaconda or Conda in all sorts of contexts because it's just a pillar of the data science and AI world. Ever since I've known about data science, I've known about Anaconda. So I'm really excited that today we have Peter Wang, who is the CEO at Anaconda, joining us today. Welcome Peter.

Thank you. Thank you for having me on the podcast.

Yeah, great to have you. Before we jump into all things Anaconda, it'd be great if you could just tell us a little bit about your background and how you eventually crossed paths with this data science world and ended up helping found Anaconda and all of those things.

Yeah, I'll try to give a somewhat of an abbreviated version of the story.

Sure, that'd be great. It's a lot to cover all around.

It's a lot to cover. It's about 20 something years here. But I started actually my, my academic background is in physics and so I was doing.

Same here.

Oh great. Yeah, Quantum information and quantum computing and physics. And when I graduated I decided to go into the software industry, join a startup. I've always been coding. I've been coding since I was a very young child and I've always loved it. But you know, just given there was the dot com era, I thought that would be, you know, good time to try my luck at that whole thing. But anyway, long story short, I ended up in Austin, Texas working at a Python based scientific software consultancy. And there, and that was kind of the early to mid 2000s and it was there that I really, I'd always been, I guess since 99, it was when I first started getting to Python, but then over the course of the 2000s when I started doing a lot of work in the scientific Python, scientific numerical computing with Python that was, you know, basically I was Using Python before Numpy existed, right, it was still numeric or num array and those kinds of things. But basically over that time, I started seeing through more and more my work and my consulting jobs that Python is being adopted in places that were outside of like what you would consider scientific computing. And so we were initially thinking, oh, this is like a cooler alternative to matlab. And then we started seeing it in business environments. Started seeing it, you know, I'd go into like investment banks and just see it being used everywhere to do financial modeling. And I'm like, okay, well this is very interesting, right? And so around just the turn of the decade, I guess, the last decade, the 2010s, you know, I think many of us in the community were having this realization. But I personally, as an entrepreneur, you know, I started realizing, you know, hey, with the joint evolution or transformational things of cloud computing as well as big data, the big data creates a demand to do bigger analysis, if you will. And traditional SQL is not going to cut it. When you have that much messy, you know, the four Vs of messy big data, SQL doesn't cut it. And then if you look at what hedge funds do, and hedge funds are usually kind of the leading edge of numerical modeling technology, they're all doing very sophisticated kinds of predictive analytics. And Python, they were choosing Python all the time. So there was that. And then I realized with, with cloud computing, it meant that every company on the planet would be able to rent computers to do their data would end up in the cloud, and they would rent computers to do massive scale supercomputing scale jobs that would have been inconceivable before because you'd have to ask it, wait five years, they might build you a data center, right? And so with these two combined transformational forces happening, I realized, aha, Python is actually a thing we should be pushing for data analysis. And so in 2012, when I founded Anaconda, of course at the time it was called Continuum Analytics, I also started the PI Data kind of community movement and kind of that branding, if you will. It's kind of a branding exercise because all the tools are basically scipy, right? And I mean, of course you have Pandas in there and a few others, you know, stats models, things like that. But ultimately it was basically scipy rebranded for a business audience. But that pushing of Python for data analysis was something that me and a few other people, I'd say we were the pioneers, we're on the vanguard and everyone else was looking at us like we were weird. It was either Hadoop or maybe it was going to be R as the scion for, you know, the successor to SaaS. But we came on the field and I was very vocal at that time, like, hey, Python's the thing, Python's awesome. And then people will be looking at me weird. But now I think we've proven that Python is a thing and it's a good and useful thing. Of course it has its rewards, but anyway, that's kind of how I came to founding the company. And we created actually the Anaconda distribution as sort of like a thing we had to do. We were interested in creating distributed computing and interactive visualization and compilers and optimizing all this and that and the other, but we couldn't get any of that into people's hands because they were still struggling to install scipy or matplotlib. So we built a distribution called Anaconda to make that problem, particularly nasty problem, even to this day, easier. And so that's just kind of continued to be a thing. So, yeah.

So to take you back just a moment, I'm kind of curious, just for perspective, kind of historical perspective, as a big advocate for Python at the time, as you're looking at this and you're kind of saying, hey, it's going this way, I'm just curious if you can take yourself back, what was it about Python then that was really driving that passion? Why was that passion not with matlab or with R or with some of the others? What was it that really grabbed you? I'm just curious how you got motivated on that.

So I am kind of a, like I said, I've been programming for a long time. I know a number of languages. I started with BASIC and Logo, as so many people do, you know, children of the 80s. But I learned a lot of languages. In fact, my professional work had been a C and I was a huge performance nerd and all that stuff. When I first included Python, it was, I think, version 1.5.2 and it was a slash post and I was like, you know, I'm tired of all these slash posts on Python. Let me finally take a look at what it is, because Perl 6 looks like it's going to take a little while to get here and I might as well play with Python, right? But when I started using it, I realized, oh my gosh, this is very nice. This is executable pseudocode. And furthermore, I can use Python to script my low level C graphics engine and be way faster than sitting there beating my head into like C templates. Which back at that time they were not very like, they were not very well supported by any of the compilers. So I could get more abstraction, I could prototype very fast and it was just pleasant to work in. I felt like I could do stuff without like slogging through a pile of like syntax. So that kind of ease of use and that friendliness, even for a seasoned programmer, professional programmer like myself, was very nice. And what ended up happening over the course of the 2000s was that you could see people who are not traditional programmers. And this is going to be a very important thing as we talk about practical AI and kind of talk about the demographics of the next generation of practitioners, the people who made Python good for science and data and data science and all this. They're not professional software developers, right? So Jupyter was born as Ipython, created by, well, I mean, Fernando, for instance, is a applied physicist. There's a lot of physicists actually banging around on the Python ecosystem. It's a trend, it's a trend, it's a thing. And then you've got like, you know, Jake Vonder Plass, another contributor to like scikit Learns and creator of Altair and whatnot. He was an astronomer. You've got like Travis Oliphant, creator of Numpy, my co founder at Anaconda. He was a double E. You know, he was a, he was an assistant professor when he made NumPy. So the tools that were built in Python for doing data science and analysis and things like that were built by people who could take what was there in the Python ecosystem, modify it so it's fit for purpose. So it was pleasant to use for what they wanted to use a computing system to do. That's very different. That's sort of the product development is coming inside the, it's inside the head, right? It's coming from within the person making the thing. So like for instance, Wes McKinney, the creator of Pandas developed when he was at a hedge fund, right? And he's like, this is the thing I need to like to like bang around my data frames. So that's a motif in the Python ecosystem that took it from just kind of this cool, fun, easy to learn language that was, you know, embodying Guido's vision of computer programming for everyone and a nicer scripter scripting system than Bash. We really, I think the scientific Python ecosystem took it to this next level of like, okay, this is the numerical quantitative computing system that we all wish we had, right? And it's unencumbered by like 30 years of legacy crap. It's on a language that's very nice to use, easy and approachable, but under the hood, you open it up, you know, this like nice little, like approachable little Honda Accord you open up. Under the hood it's this incredibly modular, like warp drive unit, right? And you can bolt on things like swig really weird pieces of software that do incredible things, automatically generates wrappers for any of these other languages, right? So you can bolt onto Fortran C, we have a just type compiler we built for it. It's a really interesting upgradable piece of kit, right. To use a British sort of term. So that's, I think what's given. It's kind of the sticking power that has now kind of, you know, now we've seen that evolve more and more. And so now it's this really cool community, almost like a standard language that's very modular and those are some of the key aspects of it.

Well said.

Sorry, that was a very long, long winded answer. But you gave me talking about true Python evangelism and I could just, you know, I can't stop.

No, I totally relate. I remember, yeah, it was late in my PhD when I was working on a piece of scientific computing code and Fortran. And I remember that was the first time that I was introduced to Python because I went to go see some of our collaborators. I spent like three weeks with them working on some, some experiments and I was like, pair programming with this guy and he's like, oh, I'm just going to run a few things. And he had this Python script around, running all of this Fortran stuff. And I remember just being stunned. I'm like, oh, why haven't I been doing this? This guy has a superpower of some kind, right? But yeah, I'm wondering, maybe moving from there. You talked about how the numerical and scientific and data science tooling that was developed in Python that was kind of developed by this sort of these groups of, whether it be scientists or people in industry doing data analysis or not really maybe the traditional kind of programmer types. Do you feel like that contributed to some of the maybe struggle or inconsistency around like managing environments and installs and like how people like, managed all of their stuff. Was it just because people, like had a bunch of different views on those things or what's your perspective on like.

Why is packaging so terrible in Python?

Right, yeah, yeah, exactly.

Well, so I can answer that question, but first I would just have sort of a meta critique of the question. Which is any system which does the number of things that Python does, I would assert, has similar kinds of software dependency issues. Okay. And even systems that don't. So JavaScript, which is all interpretable, has created an absolutely nightmarish scenario of packaging. And even though they have a vendor, the world, everything sits in the subdirectory vendor of the world kind of approach. Even though it's all just pure text, there's no compilation, you can't go and gripe about Fortran ABI specifications. I mean, it's just text. Right. But even JavaScript, everyone knows packaging is a nightmare. So even under the most non compiled sort of scenario, you end up with the packaging morass because you have an ecosystem that is able to build on each other sort of things.

Yeah, can do so many things.

Right. But in the case of Python, think about what we're talking about gluing to Fortran code. All right, well, you know, maybe Pearl has nicer packaging. Where's the Pearl glue to Fortran code and where is that being used On a million nodes?

Right?

It's not. So I think there's a thing to recognize, like the reason why your duct tape has all this crap stuck to the back of it is because it's sticky. It's duct tape. So Python is a glue language, of course, gets a bunch of cruft glued into it. And now we have a much harder problem to solve because we do speak to so many different things. Yeah, okay. Now that being said, I think fundamentally one of the reasons why Python packaging is difficult has been that packaging was treated as a second class concern by the bdfl. Right. So Guido clearly admitted very early on he just didn't think about it that much and didn't really, you know, it wasn't a big problem for him, it wasn't exciting or interesting for him. And he's not really apologizing for it, he's just saying this is kind of the way it was. It was someone else's problem to go and figure out whether it was the Philip Eby with step tools or you know, the Dish utilities, whatever it might be. It was really not a thing he was super interested in. And then when we came along, actually there's a. The very first PI data workshop I put together. March of 2012, three months after we started the company. And Guido was working for Google at the time, he came by and stopped by. We're all very excited to see him. People gave him a lot of crap about like, hey, when are we gonna get a matrix multiplication Operator. But then we also asked them about packaging. We're like, hey, can you help us get the core packaging folks in the core dev to work with us on packaging style, you know, in the scientific Python ecosystem? Because it's just been a mess for a very long time. And his answer was, look, it's possible that your needs are so exotic that you should just go do your own thing, like, don't worry about it, just solve your own problem. So we're like, okay, so we did. You have to understand, by that time the people in the SCIPY ecosystem had been fighting various multiple genealogies and multiple generations of Python packaging tools for 10 years, right? It had never really been great. And so by that time we were like, okay, let's just solve this once and for all. And we realized something very fundamental. And this is true for any system, this is nothing to do with conda. This is true for, for any system that touches compiled code. And this is now the second part of my question, the second part of my answer. The first part of my answer was that Guido didn't really care, so it was kind of festering. The second part of it is that software development, basically every single operating system that is a PC based operating system sucks. And so we have inherited the long shadow of 1970s technical debt, right? If you're on Linux, any of you guys heard of a thing called Docker or use a thing called Docker? Maybe just a little thing, right?

Love Docker.

Why did Docker come about Docker? What's that?

Because you even have rpm? Yeah, I mean, what, Linux Distro doesn't have a package manager and yet you use Docker? Because the concept of having static linkage or dynamic linkage between software libraries, even though we have a tried and true robust dynamic linker system, it's terrible, right? You go to Macintosh, okay, so maybe use Homebrew, right? That's kind of the preferred kind of package manager on there. And of course eventually Apple with App Store will kill all of that third party stuff, but. But for now we have Homebrew. Well, you start building stuff with Homebrew. What's. What if you want to have multiple different versions of libraries and you have to build framework builds of them and those framework builds are incompatible with each other and some of them need access to like the RAW GL context. God forbid you need to do any of that stuff, right? So every single system and Windows notoriously dll hell, right? Literally is the Windows DLL format, right? And they do Windows side by side, have all Sorts of things that they do. It's an unsolved problem because we've inherited the C linker and loader and ultimately that is part of the core ABI spec of the underlying operating system. So now if you're just a couple of like physics turned software nerds like me and Travis, you're like, okay, holy crap, how do we ship LLVM with our dynamic compiler? How do I ship a pile of JavaScript dependencies with like Bokeh and our interactive web viz stuff? How to do any of these things? How do we ship? For instance, people, do you guys use Jupyter notebooks or. You're familiar with Jupyter notebooks, right? Yeah. And have you ever converted them? Have you ever used NP convert to convert from a notebook to like a PDF or something like that?

Yeah, yeah.

Did you know, fun thing that it uses a thing called Pandoc underneath which has an embedded Haskell compiler. Okay, so like in order to ship NB convert on Windows, you have to go and build a Haskell runtime on Windows.

What a web of things.

What a web of things, right. But this is the kind of nonsense that we end up having to deal with. I call it nonsense, but really it's just we realized that it was more than just the C, ABI linkage, whatever, and this was back in 2012, pre Docker, but even Docker then on Windows. Not a really great story. Right? And ultimately we don't need Docker. Actually, it turns out if you build things in the right way, you can have incredibly portable, side by side, installable, native executables that work just fine. And that's essentially what the CONDA system is about. It's about rather than wrapping everything up in a hermetic sort of Docker environment, we create a very simple specification of what packages do you want. And then we have a recipe system that then has a build system behind it and that is able to build native binaries for every single platform, optimized for every single kind of hardware version that we support. And that's ultimately, you know, when it works well, it works well. When it falls over, then it can be a little bit hard to untangle exactly what the problem was. And we're working on that, of course, but that's ultimately the motivation there and why it's terrible. I think we inherited the long shadow of the 70s C, you know, linker loader. I'm looking at you. And then also we inherited some of Guido's, you know, preferences in language development. And I'm not blaming him. I love him and I'm so grateful for what he's done with the language. But anyway, you guys asked, so that's the honest answer.

What up, nerds? Jared Santo here, your humble producer. I'd like to tell you about something new. We're beta testing around Practical AI. It's a membership program which we think could be really valuable for the whole community. We call it changelog, and it's the best way to directly support Practical AI and all of the podcasts, videos and other stuff we create here at Changelog. We have big plans and ambitions for this, but we are experimenting for now to make sure there's interest. That means when you sign up today, you get Practical AI and whatever Changelog shows you listen to now, accept no ads. I guess that means this part you're listening to right now, it'll be gone. We also have some extended episodes planned, bonus content, merch store discounts, and a lot of ideas. But since it's such early days, we are offering memberships at a 40% discount for early adopters. That disappears at the end of August. So head to changelog.com/plus to join today, lock in that discount, get closer to the metal, and make the ads disappear. Once again, that's changelog.com we'd love to have you supporting us as a member.

Okay, so I'm going to start off the next section as we're starting to dive into Anaconda itself with the obvious question you've probably been asked a billion times. I get Python, Snake and Anaconda, but I am curious why specifically Anaconda and if there were any alternative names that might have been fun that you could share with us.

Yeah, well, it was. So I will give you the origin story. It was at a moment at that PI Data workshop when we were looking at trying to promote a Pythonic alternative to Hadoop. At the time, there was a distributed MapReduce system built around Erlang that had come out of Nokia and it was called Disco. And so I was like, wow, this is nice. It's a nice Pythonic mapreduce. If people really want mapreduce and we should get this to people. And then I realized I had this sort of, like, moment of, of truth. I was like, wait, we can't even ship Scipy to people after 10 years, 12 years, how are we going to ship Erlang runtimes to people, right? And so I turned around, I looked at Travel sitting in the back of the room, and I looked at him. I said, we need to create if we're going to really do this, we have to. A bunch of stuff's gonna have to go rolled in. We need to create a new distribution of Python for big data. So let's call it Anaconda, because the big snake, right? It was. No, it was like literally it happened in a flash. It was just. There was no great deliberation about this.

And when you're saying so just for listeners who are maybe, maybe a little bit newer to this world, when you're saying a specific distribution of Python, so what does that specifically change about your local environment that you're using this distribution of? Is it just around the, like using conda instead of pip, or is it a whole different Python interpreter? What are the sort of specifics when you say specific distribution?

Yeah, so the specifics are that we build the Python interpreter itself and we build the installer around it and then all the packages. Because when you build packages that have binary or C extension, module or C, they have to be built with a compatible compiler set as what you built the the interpreter with. Otherwise you seg fault, right? And then every subsequent package that has C dependencies needs to be built with the same compiler set, otherwise you end up with runtime seg faults, which is no fun for anybody. So we basically have created a normalized build system. It's like a Lego ground base plate that has equal spaced studs and is level and then you can put everything on it, but you need that first base plate. So that's what the Anaconda runtime really is. So it's a Python interpreter that we've built. And then you can either get mini Conda, which is just that Python interpreter with the standard library and the Conda package manager, or you can get Anaconda, which comes with like 250 or something, 220 packages pre built as well. So pre populated baseplate. But the idea of the Anaconda system is that using conda, you can then install packages into this that fit on that base plate and fit with each other. That being said, there's nothing stopping you from using pip. I mean, I use pip, right? You can PIP install other modules in. But if they use, if they have C dependencies or pre compiled binary components, it's better to install those with conda because then you know that those are compatible. And especially as we're talking about in the context of AI and ML and things like that, obviously accelerated hardware is a deeply important topic, right? And so you want to get the version of the package that's built for your piece of hardware. And sometimes when people go and build binaries and they make PIP wheels available, they have to build those with the lowest common denominator of hardware flags. So you might be paying $3000 for xeon processor, but only getting basically a 500$200 xeon processor worth of capability because certain flags not turned on. Right. And this is the kind of thing that is not important for like a huge number of users, but really important for some users and of course as time rolls on more and more important for everyone. So that's what that means. We can install Anaconda into a user land directory. You don't need admin permissions and you can install stuff that's all self contained in there. If you don't like it, you can blow it away with one directory remove command and it stands separate than your system, Python than anything else. So that's really what. So people use CONDA with Docker all the time. It's a very common pattern.

Yeah, that makes sense. So a lot of these packages like you're talking about, you know, scikit Learn or Pandas or Numpy or these other things, generally people think of this sort of landscape of open source data science tools. So is what portions of the sort of Anaconda system are open source and how did you, when you kind of entered into this journey, how did you go about navigating that open source landscape where you know, there's all sorts of things that have all sorts of licenses and you know, building a product around open source is kind of an interesting thing especially I think it's, you know, it seems to be a lot of things like that are trending now. But I think, you know, when this whole field was getting started, you know, there probably weren't as many examples of businesses built around like open source. So yeah, how do you all think about that and how did that grow as you were starting out?

Yeah, so Travis and I are both ardent supporters of open source because I think it leads to open innovation. So for me that's the almost like open source almost a means to an end. It's not an ends unto itself. So if something's open source, but it leads to single vendor innovation, that's still bad. And we have examples of that now actually in the burgeoning AI space. And I would encourage people to look beyond merely the kind of astroturf like, oh well this is open source. Well, all the contributors come from one company, whether it's a small one or big one. It's like, who's going to get in there? Are you going to accept patches? Can we really fork it? And you're not going to sue us for some thing? For me and Travis has been watching what the open scientific software ecosystem was able to produce. That collaboration was so generative and so amazing. We want to ensure that that would endure. So in building a business and building, trying to build business models around open source, that was part of our entrepreneurial exploration of that. Can we build a company and have a good one that fosters and sustains open source innovation? So everything in Anaconda is open source. The recipes are open source. I mean, the condom package manager is open source. That's always been the case from the very beginning, right? So we don't make our money by holding back any of that stuff. We started with a support and consulting kind of model because again, Python for data science was kind of a new thing. And there are plenty of our projects were nascent, right? Whether it's NUMBA or whether it was, you know, any of these other projects, we had a lot of consulting demands for those kinds of things. So we did that. As we shifted into product oriented mode, what we realized was that enterprise pains, especially enterprise pains addressed by software, they're not really about proprietary closed source features. They're about enterprises wanting to have roadmap transparency, having a vendor a throat to throttle when something goes wrong, you know, all these other kinds of things. So Red Hat demonstrated how you can do this in a fairly sustainable way. So we, our first product that we actually shipped was a package server that, you know, you could have an IT guy could say, you know what, I don't want GPL packages coming in here because legal, right? So I want to have all the data scientists internally be able to point to my internal mirror of the Anaconda ecosystem. But I get to blacklist all of these or zero out all of these GPL packages. I get to set which versions are available in various channels. So the Prod cluster that I manage will only ever get package updates from the prod channels and I get to flip the bid on that one. But the devs, you know, the data scientists who have the sandbox, and they want the latest bleeding edge version of something or the other, they can knock themselves out, right? And now they're not complaining that I'm holding them back from their work. So that package repository server, I mean, we still sell that today, It's a very popular product and so it addresses a deep need that enterprises have. So that's kind of how we think about the Product development. We also do have an enterprise machine learning platform just as like a domino or H2O whatever kind of thing. So we've had that, that we've been selling for a while. But I think that for us looking at kind of the growth of the packaging demand in the ecosystem, you know, the package server for us is really kind of a no brainer enterprise offering.

I'd like to actually follow up on that exact thing. And as you're looking at organizations, companies, businesses out there that are trying to find their way into data science and Anaconda being one of the major avenues on doing that.

Mm.

What is the value proposition that a CIO at some company is? It should be looking at when they're thinking about do I go Anaconda or do I go, you know, some other route? Do we mix and match? You know, because that is a question that companies are dealing with every day right now.

Right.

What should that CIO be thinking about when they're trying to decide whether they want to go with Anaconda or not?

Yeah, so there's actually not very much that's competitive with our package. Sort of a package e commercial we'll call the commercial license, you know, of Anaconda. Because ultimately what we're solving is a very unique but important problem which is the software supply chain. Right now you see a lot of companies. It's actually shocking to me because I've been involved in open source since like 95, right. In the early Linux days. But even today there are companies that are just starting to understand like oh yeah, maybe we should figure out how to use open source in a governed way.

Right.

They're starting to have that conversation at the CIO and the IT leader level. And when it comes to ML and AI, you know what it's this, the ecosystem moves so fast. It's a whole wild west of things out there. Anaconda is basically like the only company that is out there as your last outfitter between, you know, civilization and the wild west. And so if you actually want a build of Numpy or of Scikit, learn to go and run on your customer sensitive PII HIPAA data and not have it just come from some like grad, grad students, like you know, server somewhere under their desk you have to talk to a vendor, you have to have someone who will actually talk to your legal people, you know, sign us some lines. We're that we're it. So we actually are compatible with a whole host of other, you know, I talk about how our ML platform is you know, competitive with things like Domino or maybe a SageMaker or some of these other things. But our package server and our commercial license for the Anaconda distribution is not competitive with those things. It goes hand in hand with those things. So in fact we have a partnership with Red Hat, a partnership with IBM that we've just announced earlier this year, where our package server and those commercial license packages that is going out to the world via those channels, because again, there's not much competitive with that. So what the CIO should be thinking about is how do I govern the software bits that actually run, you know, like this Docker that, you know, this like 3 gigabyte opaque binary that my data scientists like intern just handed me? How good do I feel about running that in production, right? If I want actually some transparency into it, if I want repeatability, like an aerospace manufacturer was talking to me at PyCon a couple years ago and the data scientist and aerospace engineer there, he said, we have to demonstrate to the FAA that we can run these wing models 50 years, 50 years after the last plane rolls off the line. So, so it can't be some Docker file with a run command, npm, install this, or pseudo PIP update that. Right? You got to have something that you can point to and everybody up and down the governance chain feels good about. And right now that topic of open source governance for MLAI is not a broadly discussed topic. But of course you guys as practitioners understand the importance of that, especially as predictions and predictive models come under more regulatory scrutiny. So yeah, we have, I think, a relatively distinguished and unique offering in that regard.

Yeah, I'm curious. You mentioned a couple times, and I guess this is a product of kind of this shift in, in the first, the hype around data science and everybody's doing data science now, we're kind of all shifting. We all want to do AI, right? And if we do AI, then we get bigger salaries and that sort of thing. So how have you seen this shift towards AI and wanting to do AI things as opposed to sort of maybe just data science? How has that influenced the way that you're interacting with clients and the things that they want to do and the open source projects that you're wanting to support within the Anaconda ecosystem? How has that shifted things and made you think about things differently or the same, or what does that look like?

Well, it's made. Well, without getting too snarky, it's made me very cynical about the tech business press. But maybe I was already at maximum, I already maxed out my cynicism there already.

So maybe, yeah, you're just riding a plateau now.

It's flipped around now. It's now flipped around, right, Because I used a sign indejure and now I'm a negative cynicism. And the challenge is this, that, you know, as someone who kind of understands the technology in the space, I think we're very close to some things that are really very close to what we could call AI, right? So there's a real, there's a substance there. But the second order, third order far field wave of interest and hype is way bigger than what's justified, I think. But at the same time, if we, you know, I mean, GPT3 is jaw dropping, right? I mean you look at some of these things, it's like, holy cow. And then we, and there's deeper questions there. Like we have to think about this. Technology is like at an intellectual level, it's like nuclear age revolution. We can't just put this in everybody's hands. There's, we gotta be serious about how we use this stuff, right? But, but, but all that aside, I think that what people are maybe sometimes miss in that business level kind of up leveling of like AI and all this kind of stuff is that it is a ladder of needs or maslow hierarchy, right? If your business sucks at basic data management, if you can't even run SQL queries over your stuff, you're not going to get to do data science. And if you can't do data science, your AI is going to be kind of crap like you're not going to do any real AI. You spend a lot of money. No one's going to stop taking your dollars just because you have bad data. Infrastructure consultants will take your dollars all day long. They're going to produce some like clickable BI chart and charge you a couple million dollars for it. You won't have gotten the value out of it, right? But then, you know, whoever has ever gotten fired for having a bad IT project that's just somebody else's dollars, right? So again, like I said, I have a lot of cynicism about how this stuff works, but I do think that legitimately what. So to your, but in a more serious tone, to answer your question, in steering the ship at Anaconda and looking at where we make our investments, you know, we support the development of some of the fundamental tools. So we invest in things like Dask, which are next generation distributed computing in Python. We support things like numba, which give us more performance just across the board. It makes the low level libraries very performant on next gen hardware, right? So hardware manufacturers like Intel Nvidia partner with us to, you know, add improvements to the compiler. Pandas. We support fundamental development of things like Pandas. So I think my thing is this. I'm fixated on empowering the practitioners and helping them up level their data literacy across the organization. So my investment is not going to be at the cutting edge of the hype. The way I want to steer the community, the way I steer my friends who are, you know, movers and shakers in the community, is to really think about this. If we are to have this technology be something that's transformative for humanity as a whole, then it cannot become an ivory tower where there's a few acolytes who know how to use a few privileged proprietary systems to go and tell the rest of us what the predictions are, right? This cannot be how this works. It has to be a democratized transformation of how every business, every person thinks about it. In fact, an underlying thing at Anaconda is that we want to make sure that everyone gets data literacy. That's why we will always have this free capability. We don't charge for like a few hundred extra rows on this library or something like that. It's always free and unfettered access. Because I want every school kid in Bangladesh to be able to model quantitatively in a jupyter notebook. Why some hotshot politician enacted some policy, right? Everybody, everywhere. Math is empowering for everyone and this is just computational math. So from that perspective, there's a deep moral aspect to my mission and to the mission at Anaconda now for AI, ML and data science, the transformation I've seen in the field is that yeah, everyone talks about AI, but then when you get all the practitioners together, you know, we all know to put the stuff on the side, like we put the MBA speak to the side and then we all just talk about the real stuff and it's like usually data engineering, it's usually, well, software bits, right? Who's setting up the production environment, what version of Pandas are using, you know, et cetera, et cetera, GPUs that people are actually now starting to really model their hardware footprint as they're approaching data jobs, which I think is fantastic. It's what should have always been done, right? It's actually a practice that it has left behind for like 15, 20 years in the Java era. Now hardware matters again, vectorization matters again. And it's a beautiful thing. So maybe I sort of Lost my point here. I'm just like now ranting.

No, no, it's really good. Yeah, I mean, I think some of those things, things that I see like you're talking about definitely resonate with me. I definitely think like when I first got into data science, like the hardware wasn't really, I wasn't thinking a ton about it. And I was also like, you're saying, you know, shipping off that like 3 gigabyte software container to, you know, DevOps Doug and he was, you know, figuring out and, you know, hating me because it took however long to build and all of those things. So it's good to hear you, hear you talk about some of those things and I definitely see how, you know, a lot of these main components that you're supporting are really fundamental. I was just doing some speech recognition stuff end of last week over the weekend on Vidya Nemo and they are like, well, you need to install NUMBA to speed up some of these data augmentation things for the speech files, which is like a main component of the thing. It's really driving that and that really influences the, the actual AI training and the quality of that. So these sort of pre processing things and all of those are really fundamental. I don't know if this factors into your thoughts around packaging and distribution and that sort of thing. I know a lot of people are talking about these model hubs now and like TensorFlow Hub, PyTorch Hub, hugging faces, model hub. In addition to the code, there's sort of, there's the data, there's the code and then there's these, these things that are kind of weird, maybe in the software engineering world that are like these different types of data which are the serialized models and influence how this code runs. How do you think about that at Anaconda and has that conversation been going on about packaging and distributing these things or are you mostly focused on the code at this point?

We're still focused on the software supply chain, but of course I'm very exposed to the kinds of dynamics you're talking about. Right. Because I see these conversations happening in the practitioner ecosystem. And in the conversations there's a really important dynamic that's happening here that without getting too like hyperbolic and biblical about it, that data science and we'll call AI or machine learning, this represents essentially the transformation of the software industry. So for the last, probably 40 years, since the dawn of like the PC era, but even prior to that even a little bit, software developers and software engineers have been able to think of themselves As a distinguished class, like we do software, the hardware people, they're writing Verilog and taping things out, whatever that means, and making chips. They plug it in, power it on now, then we come in and we do our jobs, right? And then of course that's separate from the DBAs who are this other weird class of like Oracle licensed whoevers, right? They just sit there speaking a bunch of weird SQL all day long. So this deconstruction of the information system into hardware, software and data management is deeply unnatural. And it's actually something that was not the case. If you go back to the Norbert Wiener and to like the early cybernetics era, no one thought about it that way, right? You listen to any of these, like the founding fathers and mothers of the space. It wasn't like, oh yeah, well I'm going to focus on hardware, right? But the PC era and then everything that came afterwards and the server side on microprocessors, all these things led to this deconstruction of an information system into these three primary axes. Yeah, I guess it's a decomposition into three axes. And what we're seeing again now with data science, and certainly with ML and AI, is that we now have a synthesis. Again, we're forced to do a synthesis. We have to understand the runtime. And actually the runtime characteristics of your software is data dependent, right? That's how weird is that? You imagine talking to Java head 15 years ago to say, well, okay, Mr. Java architect, check this out. If I pass in certain values out of this database, Your code runs 10 times slower, right? That doesn't happen because you write a crud system, it pulls a row, doesn't crap and pushes a row and it's done, right? So this idea that, okay, runtime performance, so the hardware footprint is dependent on data and additionally correctness is value dependent. Can you imagine writing a unit test? And you know, we have these for models, we have model tests, right? Prior to models, it was all just code. You imagine unit test where it's like, well, one plus one is two. And so this function, my add function, works, but it only works for even integers, right? Like that's weird, right? And yet we know that when we build these AI systems, these models, their, their performance, their correctness is actually value dependent. And this is a point that, no, I don't hear anyone else making maybe because I come at it from a physicist perspective and I think about deconstructing everything to fundamentals, but deconstructing the computation concept into fundamentals for the last 40 years we've had value independent processing, right? And Jim Gray's written papers about this and people have talked about this, but like your average coder nerd doesn't think about this at all. They're like, I'm a software dev. I learned this thing, I'm gonna learn go this year, I'm gonna do something else next year. But the whole field of software is going away. It's melding into, we might call it model development, we might call it something else, but I call it value dependent or value sensitive computing. And now your management of your upstream data is as important as managing the upstream code, right? So the previous approaches to data management don't work anymore. But of course, checking in every row of a database into Git doesn't work either. So we have to develop an entire new set of practices for this new industry. All of the previous components, those axes are important, but they can be no longer be seen as separable. Right. They're now integrated. So anyway, that's, that's my welcome, thank you for coming to my TED talk about that. But I think that really that's the lens I look at all this stuff through. So it's no wonder that we have model hubs emerging. But I think the management of those things and how we talk about versioning of data, the model performance and characterizing it all that is in a nascent and emerging area that'll be fascinating to watch how that really goes as it meets production in the real world.

Okay, so a few minutes ago we were touching on deployment and I know Daniel made his DevOps Doug illusion there. And so I actually want to go back to that for a moment.

So someday I really hope DevOps Doug listens to this. So, you know, Peter, Doug was my first like DevOps engineer that I worked with when I first started out at a startup. And you know, he taught me all sorts of great things. Anyway, go ahead.

Yeah, we're going to get our first practical AI hate mail, you know, from, from every duck.

You should contact him and have him on the show.

Okay, so here's my question. I'm thinking about Anaconda kind of again in the organizational structure and thinking about, you know, having to put software together and move on to deployment. And, you know, one of the things that we're seeing a lot now is using Python in doing the data science and doing the modeling and stuff. But we're seeing kind of a move toward deploying in other languages, you know, where you may take a model and do that. And I'm kind of wondering what your thoughts are on how are you thinking like in a world where someone may for performance reasons, maybe once upon a time, or maybe still now they were deploying in Java or C, but maybe they're thinking of Google or Rust just for pure performance issues. How do you see it fitting into that in terms of pipeline? Do you think that's not necessary? What are your opinions?

Wow.

Yeah, that is a complex topic. So depending on which framework you're using and what you're doing with that, compiling down is always something that's going to be a part of the Python ecosystem. Right. I mean, Daniel mentioned Numba before, like you don't have to convince me about compile down. Right. There are times when you just need to go lower.

Right.

What pains me though is this idea that there are things that because Numba goes not down to C, it goes to machine code like we literally are generating, we're skipping a level, we're going from Python to machine code. Right. So in other places I think in case like TensorFlow and there's tools like Jax, there's a whole bunch of stuff coming out now where you can go from high level Python to much lower level runtime primitives and I think that's fine. I think when people are doing rewriting. So compile down is something different from a translational perspective than rewriting. Right. And I know that for instance, when people, just the other day someone was complaining to me about the fact that they built some models in Torch and then they have to basically to go to TensorFlow serving, they have to rewrite everything in TensorFlow that's deeply inefficient, it shouldn't have to be done that way. And so I think that the, and the problems to solve there are not monumental, I think, I think they're mostly ecosystem tooling and some of these kinds of top things that we will solve in time. So I hope that Python compiling down is not an issue and we'll keep doing that. Where people do feel a need to rewrite. I'm not sure what all they're doing in the Python. I think one of the problems with Python's explosive growth over the last hour many years is that there's simply not been enough instruction about idiomatic how to think vectorially, how to do idiomatic things in Python or do things in Python in an idiomatic way that's faster. I mean, I just see all sorts of code in the wild that's just like, oh, you know, that's. You don't have to do it that way, but you don't have time to educate everyone. But maybe we should, right? Because then what happens is you end up in organizations, businesses move with their own cadence. And now you got a data science team that's all relatively green. They write some code. It's slow. The IT person, the IT team, the software dev team is like, well, that's Python. We know it's slow. Let us rewrite it in Go because I kind of like really, really like to use Go. And they do it and they don't have to. And here's the cost of doing that. Now your iteration, your cycle time is way slower right now. When something goes wrong in production, you need to get two people on the line and then where did the translation go wrong? Right. For me, I think I still back to that point about what is the mission? The mission is to make data science literacy widespread and to empower everyone to ask questions of their world and to be able to use all of this powerful infrastructure. If to do that they have to go and hire a dev team to rewrite their stuff and Python into Go or Rust or God knows what, then. Then we failed in that mission. Right. So I do, I don't have a language bigotry of like, it must be Python everywhere all the time. But it's like I would like to make sure that, you know, Brett Victor has this concept of immediate connection. Right. I want the data scientists, when they're in a jupyter notebook or in a dev environment, when they're doing data exploration, I want them to be able to feel like they can round trip and that's on their own terms. That's a really important thing. So I think hopefully in time we'll make sure that remains a possibility for most cases.

Great explanation, by the way. Thank you.

And I think that kind of leads into something I wanted to ask you. Kind of started talking about some of the pain points that still exist between maybe data scientists and engineers or maybe this sort of gap in data literacy and these things. I was wondering, as you and I also know that Anaconda does, you know, a state of data science survey and of course you deal with all sorts of people throughout the industry. I was wondering if you could talk a little bit about maybe certain things looking back from maybe 2012 till now that you see as really encouraging things in terms of data science tooling and where we've come, and then maybe a few things that are still really open challenges that we haven't been able to solve yet.

You Mean specifically in the tooling, Is that the question? Yeah.

Or just in data science workflows, I guess that you see. Yeah, yeah.

Let's see here. In data science workflows, I think a lot of like some table stakes things since 2012 have been resolved. We have much more capable just in terms of like, you know, input handling and just like there's a lot of the basic day to day quality of life stuff for data scientists, I think has improved. People have settled on some sorts of tools as standard and they're good. So like using jupyter notebooks, which I know gets, there's a, there's a mixed feeling about notebooks. Right. And I can go on a length about them, but I think in general.

We had Joel Gruse on the show.

Oh yeah, yeah.

We've had all perspectives.

All perspectives. You know, I like to talk about threes and I think that the notebook rolls up three different things into one and in doing so unfortunately confuses the crap out of everyone because everyone thinks it's something else, you know.

Right, right.

But basically I think that at least with notebooks, the idea that people can do somewhat literate programming is way better than it was. A bunch of opaque code and then a PowerPoint. Right. So for all of the hate and all of the like, oh, line numbers out of sequence and like this, like, I don't know what, you just gave me a notebook, I don't know what to do with it. For all of that hate aside, I think notebooks are a net good because they show people ultimately again, back to that data literacy thing. It gets people excited and interested. And here, here's another thing about Python. It's good that it's Python that has that, that is in these notebooks. Python is very accessible and readable, even if you don't how to write it. So if you're a business analyst who only does Excel and someone gives you a notebook with a little bit of Python code, you can actually still inspect that Python code and say, hey, it looks like you're pulling last quarter's data instead of this quarter's data in this particular thing where you sliced it here, but you use zeroed out all of this sift code code. Why did you do that? That bit of accessibility and communication collaboration, collaboration kind of vernacular. Power should not be underestimated. Imagine it was a pile of JavaScript, how many nested braces and functional embedded callbacks and some other crap. I mean your, your business analyst has no hope of understanding what the heck you're doing right? They don't know these 20 lines are just to try to strip out some ending semicolons.

No, I, I totally agree. Yeah, there, there's actually, I've mentioned it a couple times, times on the show. This group called Masakane who is promoting new baselines for machine translation for African languages and they're involving local communities in that and they have a whole paper about this, how they like develop the community and all of that. But really central, a central piece of that was Jupyter notebooks because they wanted to involve, you know, local communities in the work. And of course, like you say, it's not like you're just going to go up to a new group of people and say, hey, you know, Click clone this GitHub repo and run this bash script and all this stuff. And so they were able to utilize notebooks and specifically hosted notebooks or colab and all of these things to really get people going. And hey, you just open the notebook, there's notes in there, there's explanation and you can just go. And of course you want people to advance from there and to really dig into things when things are, you know, there's weird behavior or something, maybe you kind of learn some new things. But yeah, I was really impressed with their usage of that. And I think that resonates with what you're saying.

Well, it's. The web has become unwritable. Right. Let's just be very clear. Like, I don't know, the last time you guys set up a website from scratch, I mean, you have a website for the podcast, obviously, maybe you had a web dev for, I don't know. But to set up a website from scratch, forget putting widgets in there, forget embedding, you know, interactive graphics. Just a website from scratch. Most data scientists won't be able to do that.

That's true.

I mean, just configuring nginx or setting up Apache, you know, getting SSL cert in there, doing all this other, you know, happy, like whatever. Like that's impossible. Right? Even for a dev like myself, it's really just annoying. So what the Jupyter notebook did, believe it or not, a lot of the value is simply making the web writable, making a writable web technology accessible for people who were not even programmers, who are not familiar or comfortable with the shell. That's the other thing that's true about data scientists. A lot of them are not comfortable with the terminal at all. Right? A lot of them are on Windows and now they can build websites on Windows with Interactive widgets running massive scale computation. Holy crap. Let's like, you know, complain about the, the Jupyter notebook at that point. It's like complaining about, I don't know, the cup holders in the Starship Enterprise or something. It's like, stop it, you're moving at warp 9. Just shut up. Right, So I think that's the thing that devs like myself who have a dev background, we look at the space of technology and it's a relatively flat landscape. I, yeah, I can go learn this language, I can do that thing. I can go grab a search, spin up that AWS credential, no problem for the average person. Every single one of these things is a cliff that's insurmountable. And this is kind of to the point of Anaconda as well, right? Advanced users, actually, a lot of advanced users do use Miniconda as their deployment technology, like I said, within Dockers and whatnot. But there's also others who are like, well, I can just, I can do my own thing. I don't really want to use this big package manager thing. But there's so many people out there learning how to do this stuff. They're on Windows, they have no idea how to what a compiler is. You know, they just want to do their jobs for their poli sci homework. Right? And so that is where accessibility, again, kind of building on the Python motif of being accessible as a language, want the tooling around this to be accessible. And this is. Now to answer the second part of your question, where have we fallen short? I think that as the technical space of data science, machine learning has, you know, grown up, put on a suit, got a real job, we've got software developers coming in saying, hey, I'm going to retool as an ML engineer, I can learn the stats. And they can, they're smart people, they can do all these things. Part of what's being lost nowadays that I see in the more modern tooling is there's kind of a taste is lost and the taste of making this accessible for people who are not ops geeks, I think some of that drive has gone away. So when I go and look at the documentation for any kind of ML framework, and this is not to put a ding on them. I mean, this is, you're doing complex orchestration, there's going to be some work involved. But just in general, that sense of like, how do we make this dirt simple for that poor atmospheric scientist, trying to model hurricanes, trying to understand climate change, how to make it simple for these guys Trying to build better translation, machine translation for African languages and they're out in an African village trying to get the locals involved, how to make stuff really easy to use for them. That kind of thing is, I think, being crowded out a little bit. Right. That kind of sensibility and taste is gone, I think. And it's unfortunate. There's also a willingness to embrace really big corporate open source, which is not. Again, I salute the companies for open sourcing their technologies, but I think the open source ecosystem around scipy and whatnot never really had some of the corporate open source like hegemony kind of thing happen to them that happened in the Linux space. So they've never seen weaponized open source. And I'm one of the few people trying to go in there and like raise the banner and being like, hey guys, recognize there's like community, grassroots kind of, you know, open source, community innovation. And there's like big companies saying, here's our big old toolkit, a million man hours and it's all yours. Use it, please. And by the way, it runs best in our cloud framework. Like that's, you know, I don't know how I feel about that. Exactly.

So it's more than a license, right?

It's more than the license. It's about community innovation and open standards. Yes, absolutely.

So as we finish up, I just kind of wanted to get your sense of what, what does the future of Anaconda look like? What's in your mind that you'd like to do that you haven't gotten to yet? What should your users be looking forward to at this point?

Yeah, so that's a great. Thank you, great question. So we've been working a lot of infrastructure technologies for the last several years trying to help get the commercial adoption of data science and open data science successful. Right. Which is why we sell like a package server, things like that. We don't want it raising exception saying you cannot use thou shall not use Python because we're a Java shop. Right. I wanted to put that argument to rest. So I think we've been pretty successful with some of the things we've done to help smooth those things out. Going forward though, I want us to kind of lean more into the practitioner community, help the community and thought leaders and practitioners, diverse voices right across culture, across background and whatnot, to surface organically in the community and really drive a conversation about the practice of data science and quantitative like quantifying our world, modeling our world, predicting our world, but doing it in an open way, in an ethical way. And in a, in an intentional way. I don't want us to do. I don't want data science to end up where social media has, where it's like oh, we accidentally destroyed democracy crap. You know, like you want, you want data science and predictive analytics to be sort of like hey, we know exactly. Well, we know we're going into it eyes wide open. So I want to create a community community tools. So tools for ethical practice of, of data science as well as then facilities site, you know, kind of some of the next generation capabilities that are practitioner facing. So we're going to be, you know, look forward to a lot more of that. We're trying to engage with the user community a lot more. We'll be revving our product offerings there as well as some of the things that we'll be standing up on the website itself on the Anaconda cloud. So really excited about those kinds of things. It's beyond tools at this point. It's about people. It's always been about people but now the emphasis is kind of coming back around to being about the practitioners in the community.

Awesome.

Well, thank you so much. I think that's a great way to end. We'll of course link to a bunch of Anaconda things in the show notes so make sure and check those out. Connect with us on Slack and LinkedIn and Twitter. Let us know how much you appreciate Conda over the years and you know what they're doing. Thank you so much Peter for joining us. It's been a pleasure.

Thanks Daniel. Thanks Chris. This has been a lot of fun. Thanks for listening to my rants.

It was fun. Thanks.

Thank you.

Foreign don't forget we have a giveaway going on in celebration of episode 100. Enter for your chance to win some awesome AI hardware from Nvidia intel and Google plus, practical AI and pachyderm swag. We're giving away three bundles so you have a good shot at them. Check your show notes for details on entering. You have until the end of the month. Speaking of Pachyderm, a little birdie told me they have a big announcement coming soon and you should join their Slack channel to stay tuned. Learn more about that@pachyderm.com thanks to our longtime sponsors for their continued support. Shout out to Fastly Linode and Rollbar and to the mysterious Brake Master Cylinder for these awesome beats. That's all for now. We'll talk to you again next. Sa.