Elixir meets machine learning

Big thanks to our partners Linode, Fastly and launchdarkly. We love Lino. They keep it fast and simple. Check them out@lino.com changelog our bandwidth is provided by Fastly. Learn more at fastly.com and get your feature flags powered by LaunchDarkly. Get a demo at launchdarkly.com this episode is brought to you by our friends at O'Reilly. Many of you know O'Reilly for their animal tech books and their conferences, but you may not know they have an online learning platform as well. The platform has all their books, all their videos and all their conference talks. Plus you can learn by doing with live online training courses and virtual conferences, certification practice exams and interactive sandboxes and scenarios to practice coding alongside what you're learning. They cover a ton of technology topics, machine learning, AI programming languages, DevOps, data science, cloud containers, security, and even soft skills like business management and presentation skills. You name it, it is all in there. If you need to keep your team or yourself up to speed on their tech skills, Then check out O'Reilly's online learning platform. Learn more and keep your team skills sharp@orali.com changelog again o' rally.com changelog. What's up? This is Adam Stakoviak, Editor in Chief here at Changelog and you are listening to Practically I, a weekly podcast that makes artificial intelligence practical, productive and accessible to everyone. This is where commerce conversations around AI, machine learning and data science happen. And today we have a special episode, a crossover episode from the Change Law podcast for you. On a recent episode, Daniel White Knack joined Jared Santo talking with Jose Valim, Elixir creator, about numerical Elixir. This is Jose's newest project that's bringing Elixir into the world of machine learning. They discuss why Jose chose this as his next direction. The team's layered approach influences and collaborators on this effort and their awesome collaborative notebook that's built on Phoenix Live View.

All right, I'm joined by Jose Valine, creator of Elixir and frequent guest on the Changelog. I think this is your fourth time on the show. Welcome back.

Thank you. Thanks for having me again.

Excited to have you. Lots of interesting stuff going on in your neck of the woods. And I'm also joined by. Hey, that's not Adam, that is Practical AI. Co host Daniel Whitenack.

What's up Practical Al. Yeah, yeah, Practical AI. Sometimes it looks like with the font on zoom, it looks like Practical Al.

Yes.

So when we record on our podcast, normally I'm known as Practical Al.

Well, welcome to the show. I'd have a Tool Time reference. You know, you'd be my Al Bundy for the show, but that would be too old for most people to get that one. Yeah, and I'll just say you can be my Adam, I'll be your Chris Benson, and we'll co host this sucker. How about that?

That sounds wonderful. I'm excited to be here.

Well, I had to call in the big guns because I know very little about this space. In fact, almost everything I know about the world of artificial intelligence I learned from producing practical AI and by listening to Practical Al do his thing each and every week. So that's why Daniel is here. I do know a thing or two about elixir, but nowhere near as much as Jose. And here we're at the intersection of those two worlds. So kind of an exciting time. And we're here first to talk about nx. So, Jose, what is this NX thing you're here? Tell us about.

All right, so NX stands for numerical Elixir. And back in November last year, we started working on this. I can tell more about the story later. But the important thing is that in February we finally unveiled nx, which is a library, but also this idea of a collection of libraries to improve elixir so we can start doing machine learning, data science, numerical computing, and so on. So I'll just give an overview of what we have out so far so everybody's on the same page and then we'll expand on that. So we start with nx, which is it's a library itself, is the idea in the library itself. And the main abstraction in nuxt, as you would expect, is multidimensional tensors. So you can do. When I announced the nuxt, one of the things that I did was that I gave a talk, and in this talk I built a MNIST classifier, neural network classifier for the MNIST data set from scratch just using nx. And so, you know, you can work with multi dimensional arrays, tensors, and for those who are not familiar, why multidimensional arrays and tensors? One simple example I like to give is like, for example, if you take an image, if you need to represent that image, if you need a data structure to represent that image, you can represent that with tensor, and it's going to be a three dimensional tensor where one of the dimensions is going to be the height, the other is going to be the width. And then the third dimension is for the channels like RGB and so on. And then if we can represent the data like these. You're going to send this tensor for data networks and through neural networks, and at the end it's going to tell, hey, is this a dog or a cat or more complex things? So that's where we started. That was the first building block that we built. And one of the things that people ask a lot is that, you know, like Elixir is a functional programming language. And functional programming language, they promote a lot of immutability. They promote immutability a lot. Which means if you have a multidimensional tensor, like if you have a 50 megabytes image and you need to do something with it, you need to transform this image. Each transformation that you do is going to copy the whole image in memory and do a new copy. So you are locating 15 megabytes every step along the way. So to solve this, what we did, and this is an idea that we've seen elsewhere, for example, in the Python community, we have jax. So a lot of inspirations in NX come from jax. So the way we solve this in NX is that we have this thing called numerical definitions. And what numerical definitions are is that they are a subset of Elixir that can compile and is guaranteed to run on the gpu. And that's how we can have numerical computing, elixir, machine learning and neural networks. Because we can effectively look at your Elixir code and say, hey, I'm going to get all of these, compile it, run the GPU and it's going to be really, really fast. So those are the two building blocks. We can come back to these and talk a lot about those things later. And then we release two bindings for nx. So one is exla. EXLA is a binding for the Google xla, which stands for Accelerated Linear algebra. So if we're using TensorFlow FL, what is running the things in TensorFlow is Google XLA. So they're using Google XLA to compile it, to run on the gpu, to run on the CPU as efficiently as possible. So we have bindings for that. We are also now working on bindings for Pytorch, to be more precise, Lib Torch. So Pytorch for Facebook, they have the Libtorch, which is the C library. We are wrapping that as well. And two months later, so that was in February, we released two other libraries. So one is Axon. So we started with the building block, which was tensors, multidimensional arrays, numerical definitions. So we released Axon, which is a high level library for building neural networks. And we just announced Live Book 2, which is interactive and collaborative code notebooks for Elixir. So that's kind of what we have released in the last two months. And it's just the beginning. There's still a lot of things we want to do, but we are really starting on working on this ecosystem and building it up.

So, Jose, I'm curious from the AI perspective and I, I'm going to have to admit for listeners that I know almost nothing about Elixir except what I've learned on the Changelog podcast from you in previous episodes. So I'm curious like from the community standpoint, what was really driving your motivation to spend so much time on these things? And I see like, and we can dig into the individual components, but like you're saying that like the main components that I think could make this very functional, it sounds like are there and are being built. But from the community standpoint, were people requesting this? Were people trying to sort of roll their own sort of neural network stuff in Elixir from your perspective, what sort of led up to that side of things?

Okay, that's a great question. And to give some context. So one of the things like going way, way back, it always started because of like the Erlang virtual machine. The only reason that Elixir as a programming language exists because of the Erlink virtual machine. And the Erlink virtual machine was built by Ericsson, which is a telecommunication company for building concurrent, distributed and fault tolerant software. I'm not going to expand on that. You can check Elixir on the website, but all these things for my love for their Levertel machine. So when I created Elixir, I was like, I want to have as many people as possible building on this platform because I love it and I think other people, they are really going to love it and enjoy it too. So I've created Elixir and I've always thought, I think in terms of programming languages, I really think that Python is a really stellar example of tackling a bunch of different problems. So I always had in mind that I want that for Elixir and for their virtual machine, for their language ecosystem. I think we can grow diverse to solve all the different kinds of problems. So I come from a web background. I was a member of the Railscore team, almost like a life. When I started with Elixir, I had this obvious web background and that was one of the first, let's say, dimension that Elixir took off with the Phoenix web framework. So people started Using Elixir more and more for the web. Elixir was already a good natural fit for building distributed systems or anything regarding the network due to the Erlang heritage. But it was like, I always want to try to expand this and the first time I expanded this was back in 2016. We released abstractions for data pipelines and data ingestion. So if you need to consume queues and you need to do that very efficiently, we released libraries for that. And that brought Elixir to a new domain which was data processing. And there are some very nice use cases on our website. For example, how change.org for example is using data abstractions that we wrote back then to process. Because when somebody, if you have a petition that 1 million people sign, you need to send them an update. Now you have to send an email to a million people. How are you going to do that? So we start that segment and then the community start to grow. So people start bringing Elixir in the early virtual machine for embedded. So there is the Nerves framework. People started bringing that to auto with streaming. And then there's always the question like, you know, why not numerical computing, why not machine learning? So I always had this interest like, you know, I feel like it's part of my responsibility, part of my job to try to broaden the domains and the areas of the language. The community is also doing that a lot for a bunch of areas. But you know, if there's something where I feel like, hey, this is a good opportunity, we can do it, then why not? Then let's do it. And this always started just to finish giving more context when Prague, Prague. I always had this interest actually like my thesis, my master thesis was in text classification and by using test classification, but that was like 11 years ago. So that was deep learning. We're not talking about deep learning at the time yet. I think everything was still support vector machines or kind of state of the art. But I never fall back. But I always had this interest. So in October last year, Prague Prague announced a book which is Genetic Algorithms for Elixir. And then I was like, hey, apparently there's somebody who knows things about AI and machine learning in the Elixir community. He's shun show more. I sent him an email and I was like, hey, I think the platform could be good for us to do everything in machine learning. He said like, I agree, let's work on it. And we started working on it. So it was basically, it's kind of like, why not? If we can make it happen, let's make it happen. And we will try to figure out how we are going to let's build this. And then later we will continue working on how to package and how to sell this to people and say like, hey, what are the benefits of having those two words like joining together and working together.

So if we stay big picture, but we do a bit of a comparison. Trying to understand exactly your aim here. If I was a happy numpy slash Pytorch like that Python data scientist kind of a person, are you hoping that maybe someday the NX based and Elixir based tooling would draw me over to Elixir? Are there aspects of it that it's going to be well positioned better than Python, or are you more just saying, well, let's bring this area of computing to existing Elixir and hope to give them more tools, or are you also thinking from the other direction?

Honestly, I never tried to look at it that much ahead. So for me, my goal right now is that, for example, imagine that you're building an application elixir and then you need to resort to do something with machine learning or data science and like, oh, I need to go to Python to solve this problem. If we have a tooling so you don't have to go there and you can stay within the community, I would already consider that a tremendous victory just because that was not an option in the past. So if people, they're starting to make this choice, I would already be very, very happy and I would be like, you know, like mission accomplished.

Gotcha.

And then we'll see baby steps.

Daniel, what tools do you use in your day to day work?

Yeah, I like the framing of how you just framed it, Jose, because actually my tool set, my team's tool set, we develop models in Python using TensorFlow and PyTorch. But typically in terms of the products that we're building or what we're developing, we're developing either API servers or something. For the most part we're doing that in Go. So a lot of times what happens is exactly what you were saying. So we're happy writing our API handlers in Go and everything's nice and wonderful and then we basically just have to call into Python to do an inference. Potentially. Now there is new stuff coming onto the scene in the Go community as well to try to support that same sort of workflow where I would love to not do that. If I was working in Go and I didn't have to call into Python, that would be super cool. And I think that's still developing. So I totally get what you're saying. Like, if you're working in Elixir, then it would be great for those developers to not have to do this sort of awkward call into Python for inferencing. It's awkward. And always managing that and monitoring it and all that is, is sort of dicey. Also though, I think that there is this sense in the Python community, or well, I'll say the AI community, that Pythons sort of consume this whole world, but I don't think necessarily out of like a particularly one good reason why it should consume that whole world. Because it's kind of like all these scientists or like grad students working on computational science and working on AI, they're like, well, all our stuff that our advisor wrote is in Fortran. I don't want to like write Fortran, so I'm going to write this Python stuff. Like that wraps around my Fortran. And then like, people just start writing Python a lot because it's like pretty easy to get into and they. So they do all their scripting in that and eventually like the science world just sort of started latching into Python and building things there. I don't think it's necessarily like the best tools for AI will be built using Python, actually. I think like a lot of my frustrations in life are because of, you know, working in Python. And I'm not trying to bash that because it's. It's also great, like you're saying, I think there is an opportunity for both sides of things, I guess, is what I'm. What I'm getting at.

That's interesting to hear that, Jose. One of the things that you did with Elixir, which I appreciated, I think a lot of people appreciate it because you got a lot of people loving and using the language right, is you took all these things that influenced you and that you appreciated and you brought them together with the beam. Your love for Erlang was the reasoning. But then you went to your language design and you designed the language and you pulled in ideas from Ruby and ideas from Perl and ideas from functional languages. I'm not sure which ones, but you've told this story before and you can probably reiterate all your influences and you kind of made this, what I think is a really beautiful language out of it. But it was based on your history, your knowledge, your taste, what you liked. Here you are doing numerical stuff and you're doing data sciencey stuff. And I just wonder, how do you acquire that taste? How do you acquire that Knowledge. Do you just know every domain very, very well, or how did you learn this stuff? I know you said back in school you were doing some of this stuff or statistical things, but how have you come up to speed on what would be an awesome way to do numerical elixir?

Yeah, so this time it has really been shown in Jekyll. So all the part of deep learning and how things should work. Sean, he really the one leading it. But the main seed that led to this was actually Jackal Cooper. So when I started talking with Sean by mail, before we started working together, I sent a tweet, I don't remember, but it was asking about some references. And then he pointed me to the JAX library in Python, which a lot of people are taking it to be the next big library, potentially replace TensorFlow. That's what some people speculate, but it's from Google, there's a lot of traction behind it. And then I was reading the docs for jax. So we were saying, hey, Elixir is a functional programming language. And as a functional programming language, everything is immutable. So work with multidimensional data would actually be very expensive. But then I'm reading the docs for jax, which is a Python library, and then they have quotes like, JAX is intended to be used with a functional style of programming. And then they say, unlike numpy arrays, Jack's arrays are always immutable. And then I was like, what is happening here? So it was like this reference, like, hey, it functional, right? So that, like my spider senses, they were like tingling, like, okay, wait, wait, wait, wait. So there is something here. That's when Sean and I, like, we jump with both feet and we're like, okay, there's really something here. And the whole idea in there is, because the way that JAX works and the way that numerical definitions in NX works is that when you are doing all the operations in your neural network, like, hey, we need to multiply those tensors, we need to calculate a softmax, we need to do the sum. When you're doing all those computations, you're actually not doing those computations at the moment. What you're doing is that you're building a computation graph with everything that you want to do in that neural network. And then they get this computation graph. And when you call that function with a particular tensor, with certain dimensions, a certain size, it emits highly specialized codes for that particular type of tensor for that particular graph. And that's why everything is functional, because what you're doing is building a Graph, you're not doing any computations, and then you compile that to run the gpu. When we saw this idea, it was like, hey, everything can be functional. And when it started, it was like a bunch of happy accidents, a book being published. So I like to say I really have a thank you for Pragproc, because if they did not publish this book, if somebody read the proposal that SEAN sent to PragueProc and say, hey, we do need genetic algorithms book for Elixir, maybe none of this would have started. And then somebody pointed us to jax. So it was all those things happening. And that kind of gave me a path for us to explore and come out of this. And I really think so. I said, we are going to start working and as we build the tools, we are going to try to find what advantages Elixir can have compared to other programming languages, for example. And it turned out that, as I keep saying, what I thought would be a negative aspect, which is immutability, really turned out to be a feature. And it's really interesting because there are some pitfalls in jax, for example. So if you go to the JAX documentation, they have a long list of pitfalls. So there are some pitfalls in the JAX documentation that they do not happen in the Elixir implementation in nx, because everything is immutable. So the way that JAX works is that in Python they call it the tape pattern. So basically, as you're calling methods in an object, it is recording all the methods that you call. In Ruby, we know it as method missing, but there are some operations in Python that they cannot be recorded. So for example, if you are setting a property, for example, in the object, or if you pass that object to a conditional, you don't know that that object is being used in a conditional, so JAX cannot record that structure in your code. So they have some pitfalls like, hey, you have to be careful. Or if you have a for loop, if you have a for loop in jax, what it's going to do is that it's going to unroll the loop and that can lead to very large GPU code. But in the next, everything is immutable. So we don't have those operations in the first place. And because we have macros, I can actually rewrite the if to be a if that runs in the gpu. So this is really cool. So in nacs, when you go to the numerical definitions and you look at the code, that code, no pitfalls, is going to run on the gpu, is going to be sent to the gpu. It's effectively a subset of Elixir to run the gpu. So yeah, so you know, it started with this small tip and then it kind of spread from there.

This episode is brought to you by Snowplow Analytics. Snowplow is the behavioral data management platform for data teams. Maximize the value of your behavioral data using Snowplow Insights, a managed data platform that's built on leading open source tech leveraged by tens of thousands of users. Capture and process high quality, quality behavioral data from all your platforms and your products and deliver that data to your cloud destination of choice. When marketing needs to make data informed decisions, when product needs next level understanding, and when analytics needs rich and accurate data. Snowplow is a solution for data teams who want to manage the collection, processing and warehousing of data across all their platforms and products. Get started and experience no plow data for yourself@snowplowanalytics.com Again, Snowplow Analytics.com.

So sitting on top of NX is Axon, which is NX Powered Neural Networks. When they give us the skinny on that tool. Jose.

Yeah, so it's pretty much what the name says. It's neural networks built on top of an X. And so Sean is the one. So a lot of those things, Sean's the person behind it. So Exxon, Exla, it's all Sean's work. And what he did for Exxon is that he built all of the building blocks of a neural network he built just using functions. They are regular numerical definitions, they are regular numerical definitions, and numerical definitions are regular elixir functions. So he just built a bunch of functions and then you can compose them together to build the neural networks. And so he built all of this. It was really funny because I think we can still find it in the repo. He created the initial issue, which I think had like 100 checkboxes, which was just like all the functions that you use, like all the initialization functions, optimizers, layers, activations, everything that you have in a neural network that you usually use. So he listed all of those, then he implemented most of those and then he came up with a higher level still inside Axon, higher level API. So you can say, hey, I have a neural network that is going to be this dense layer and this convolutional layer and this activation and this, and I want to train it and you're done. So the same level of API convenience that you would expect from Keras or from Pytorch is there in Exxon, but the building blocks as well that's what Exxon is about. It's a little bit out of my reach of my understanding. And it's kind of funny because I can run the basic examples, but I still don't have a gpu. And then if you get a convolutional neural network, if you're going to train with audit, GPU is going to take a lot of time. So I cannot run some of the examples. But Sean, he added already a good amount of examples to the repository. So we have some very classical data sets that people use in machine learning, like mnist, cifar, I don't know if I'm pronouncing those correctly, Daniel, but you probably know what I mean. The fashion, MNIST and so on. And he has examples of. And then no algorithms like Resnet and this kind of stuff. And there are examples already in the repository and for those things running Elixir and compiling and running on the GPU, which is very exciting.

Don't you have a GitHub sponsors or a donation button, man? Let's get this man a gpu. Someone's got to get you a gpu.

Yeah, I know, right?

Come on. The world would be a better place if Jose Valim owned a gpu. I'm going to put it on record.

Yeah, I was really like, in just an aside, I was like, I'm going to buy a Linux machine then so I can have the gpu. And then Apple came out and was like, oh, we have TensorFlow running on M1, but they released just like the compiled executables and not the source code. So I'm like, do I buy a new machine that is going to take space in my house? And then three months later Apple is just going to. The thing is going to be merged in TensorFlow and I'm never going to use it again. So in this way I'm suffering for this. I'm like, you know, should I invest on this thing or not?

Well, you've come to the right place. This is Daniel's expertise right here. This guy, he builds these things in his house.

Unfortunately, it's all crazy right now. I know we ordered a server and we had to switch the GPUs because of like, I don't know if you saw Nvidia's. They kind of got mad that everybody was putting consumer cards in their enterprise servers and so that all got switched up, which I understand their business, but yeah, that whole world is crazy right now in terms of actually getting your hands on. On something as well.

Supply shortages and everything.

Yeah, yeah, it's insane just scrolling through this like I'm, I'm pretty excited to, to try this on my, you know, little workstation with a gpu. I think it's cool that again, I'm coming not from an Elixir standpoint, but I recognize the API, like it's very keras, like this high level API that you're talking about where you're. I've got a dense layer, I've got, you know, a dropout layer, whatever it is that like instantly makes sense to me. I feel like I could take this API and create like my model definition fairly easily. And I really like that. Being a Python user and coming from that outside world like it, it makes me want to, to play with this. If it was a totally like different sort of looking API, I think I would have a, I would be sort of nervous to dive in. But I also see like, you have your model struct, you have your layers, you have your high level API and you talk about it like it's just an Elixir struct and so serializing it to multiple formats is possible and talking about the model itself. So I don't know a ton about Elixir structs, but this sort of serializing it to multiple formats is really like interesting to me because at least from my perspective what I'm seeing is a lot of sort of push for interoperability in the AI world where like people like publish their model that they wrote in Pytorch on Pytorch Hub and like, then like I'm over here with TensorFlow but I can pull it down and like convert it using something like Onyx Tools or something and use it in TensorFlow or maybe there's all sorts of frameworks out there and I think people are generally realizing it's not going to be one that wins the day. But interoperability is really, really important if we're going to release models and expect people to be able to use them. So I don't know, was that sort of factoring in your mindset as you're thinking about how to represent models in Axon?

Yeah, definitely. When Shaw was working on it from the design, we were thinking how can we get on X model, load that into an Elixir data structure so we can get that and send to the GPU and have that running on the gpu and it goes back to what we were talking about a while ago that I think the first users of this, maybe I'm wrong and I'll be Very glad to be wrong. But I think the first users they're going to be, hey, we have our data scientists that are already super familiar with this tool in Python that is very productive, very useful for them and it's harder to convince them to migrate. But hey, we are running Elixir in production and I just want to bring that model and run directly from Elixir. Elixir. And I think that's very important for that use case and the whole purpose of interoperability. One of the things that I think it's really worth talking about that I think with these ideas. So a lot of people, they think about Elixir, they think about web, but Elixir is also really good thanks to the nurse framework for embedded. And I think there is a lot of potential in this area of having machine learning neural networks running on the edge and nerves can help with that. And that can be an interesting application as well. And that requires kind of the same ideas because you're not going to train on the device, you need to build the model elsewhere and do all these steps and then bring that into the device. Serialization is there and I think it's a matter of time into a lot of the things we are working on them as you know, it's like we also started a machine learning working group in their language Ecosystem foundation so people interested in this. So it's something that we plan to work. But if somebody is really excited about this. So if you're listening to the show like hey, I want to try this out and maybe I can implement onnx Serialization and you'd like to work with us and the pr, it's definitely welcome. We can have a link to the Erlang Ecosystem foundation, the machine learning working group in the foundation. So we have a Slack people can join, can talk to us and there's a lot of work to be done and this realization is definitely going to play a big part of it.

So how long have you both been working on Axon? Because it just seems like there's so much implemented like you were talking about, hey, we need all of these different layers implemented that people know about typically I see libraries like maybe that have a new API for machine learning or something. It seems like it takes them so long to sort of add operations and add support for different layers and such. And I'm wondering what was your thought process and approach to building this in a way that you could come out of the gates supporting as much of.

That as possible to give you an idea. So Sean has been working on it, on his free time. Okay. And he started working on Nexon as soon as we announced the Nexon. So he has been working on it for two months on his free time. And it already has a bunch of stuff. If we check like the Readme, you know, it already has the cool, I'm not going to be able to see everything but the dense layers, dropout convolutional layers, a bunch of optimizers like 7, 8. So he has been able to add those things really, really fast. And I think one of the reasons for that is because the foundation are just functions. You're just building functions on top of functions, so it's very easy to compose. The other thing is also that I think one of the reasons I'm speculating here, to be clear, I think maybe one of the reasons why some of those libraries, it takes a lot of time for them to implement a layer is because they are implementing everything they are going to, maybe like from Python all the way down to the C code and implementing or C code and implementing that. While for us it's a very layered approach where Axon just works about an X and X is the tensor abstraction and then we have the Tensor compiler stuff that compiles for XLA and working at those different layers. When you're working at Axon or in Axon, you are really at the high level. You're not, not really worrying about C, C, any of that. You're just saying, hey, what are the tensor operations that I need to do? And I think that's why he was able to be so productive in building all of those features in this short time frame. And I think adding new activations, layers, they're relatively straightforward. What takes more time in discussion is when we need to change the topology because that requires to think about how the struct is going to represent that. So for example, if you have a GAN or a recurring neural network now you have to think like, oh, if it's recurring now, you need to get the data, feed it back inside. So you have to think how you're going to model that. But it's mostly just at the high level representation. So that's kind of how things have been structured.

Yeah, I cloned the repo and his first commit was January 25th of 2021. It's pretty amazing with a few to follow. And it was funny because like the first commits are like, add some functions, more functions, adding some even more common functions. So he's just like cranking out these functions like you said. Yeah.

So that was in January. Okay, yeah.

So a couple of months.

Yeah. But while working on that, he was still working on XLA and the next with me. So we started in November. So in November it was Sean and I, we were working part time, so it took us about three months to release Annex and xla. And then Sean, he's still working with Annex and xla and then he's focused after we announced it in February, changed to be on Axon until we announced it. And now we are probably all like kind of going back and forth between projects, so. Because there's still a bunch of things that we want to build in Nack. So one of the things that I really want to work on is streaming, because Elixir is really good for streaming and I want to have a very good abstraction. So, you know, we can start streaming data to be inferred like into the GPU so you don't have to load everything into memory. Or for example, if you have a webcam or a camera that it's your embedded device or you are getting from WebRTC or something like that, and you want to send that straight to the GPU and stream it. So we can do all this kind of stuff, interesting stuff that I think we can do. So, yeah, so we are going to be jumping back and forth on that.

I think it speaks to the power of a solid abstraction too. And like a layered approach when done well, when you get to those higher layers, like you said, unless you have to change the topology, if you're just adding and building on top and not having to drill down through each time, then you can move relatively fast. There's probably also an aspect of this where it seems like Axon's API is trying to be familiar. And so a lot of times, at least for me, the slow part of software is getting that API figured out and rewriting that API so that it's better. And maybe there's a step up because of all these other projects that have come before for that makes it familiar to Daniel and other people who are working in this world.

Exactly. That's a very good point. And I think on the Excel side, one of the inspirations, I think there is a project, think AI in Python, which is a functional approach.

Yeah. There's a team in Europe that writes the Spacy library, which is an NLP library. And I think that their main, like, backbone for that is thinc.

I see. Yeah. So that has been one of the inspirations as well. And I think there is PI lightning or Lightning torch or something. Like that, that has also. So yeah, that's a very good point, you know, so if you can look at what people are doing, say, hey, this is what I think it's good, this is what I think it's going to fit very nicely at what we do. That speeds up the process considerably as well.

Changelog is the best way for you to directly support practical AI. Join today and unlock access to a private feed that makes the ads disappear, gets you closer to the metal, and helps sustain our production of practical AI into the future. Simply follow the changelog link in your show notes or point your favorite Web browser to changelog.com Once again, that's changelog.com.

I mean there's just such diversity in the AI world in terms of the types of models that people are building, but there is a fairly consistent, like if you look at the implementations, whether it's TensorFlow or PyTorch or, you know, these other frameworks, you can kind of get a pretty quick sense of how they're building their architecture, looking into the source code and I mean, I'm just looking at some of the layers that are implemented in Axon and like I said, I think you've done a good job at like, I don't know how to read Elixir. I can sort of get the sense of what, what's happening here and I think that's a testament to like, yeah, like following some of the inspiration, the good inspiration that's already out there in the world. And also I think it'll be easier for people maybe that do want to jump and, you know, experiment in Elixir from the Python world and they want to add their own cool layers into Axon, it's going to be a lot easier for them to jump in and do that. I think if they feel like they're not in a total foreign world, they recognize some of these components and all of that. So I definitely think that that's a good call. I know that some of like data science AI world kind of operates with a weird set of tooling that includes these things like called notebooks and other things. I know, I saw there's even some functionality related to interactive coding and cells and that sort of thing too, isn't there?

Yeah. So there is a separate project. Another person has been working on this project, Jonathan Kuosco. When Sean and I started talking like, hey, we want to build this foundation for machine learning, numerical computing and then we mapped a bunch of things that we have to do and there are a bunch of things that we have not started working on yet. So for example, we don't have an equivalent to data frames. So that's an open question that has to be solved. We don't have plotting libraries yet, but one of the things that we want to do was this idea of the interactive and collaborative notebook. And to give a bit more context, Daniel so we have the Phoenix Web framework in Elixir and the Phoenix Web framework, I think two years ago launched something called LiveView, which makes it really easy for you to build interactive real time applications but on the server. So without having to write JavaScript, which if you're not a JavaScript developer, that can be a plus. And because the logic is on the server, it allows you to do like collaborative. Because if you have multiple people collaborating on a text, the server is the one that knows where people are, what they should do, how the text should change. So it's really good for building these kind of applications. The elevator pitch is not correct, but the one line somewhere that you can say is like react. On the server is where you can think about things like I want to do this. We want to build this notebook thing as well, which we called LiveBook. So that's the Livebook project. And the way it started was very funny. So we have a project called xdoc which generates documentation for Elixir and we're really proud of it. We think that our document documentation just looks great and it's standardized. All the projects in the community, they generate this documentation with xdoc. It has a bunch of great features. And somebody sometime ago open up and say, hey, this project is using jQuery. JQuery is huge. We probably don't need to use jQuery anymore. So somebody opened up this issued issues tracker. I was like, sure, sounds a good idea. If somebody wants to do it. Then out of nowhere somebody sends a request. They didn't ask if they should do it, they just set up request replace jQuery by JavaScript. And I was like, this is great. I reviewed the code, the code was flawless. I reviewed the best of my power CCT JavaScript and then I went to check and then I was like, oh, Jonathan, he lives in Krakow, which is where I live. He goes to hh, which is where my wife studied. This is very interesting. And it's like, oh, he has some Phoenix experience and he's still a student. And I was like, you know what, maybe he wants to, to work with us on this LiveBook thing. So I sent him an email like, hey, you want to talk at this time we had not announced the next yet, but we have announced some benchmarks comparing code running on the GPU and not on the GPU, which was like 4,000 times faster or something like that. And then I told him like, hey, do you want to work with us? And then he's like, sure, but I'm a student, like, no problem, you're going to work part time. So he started in January working on Livebook and the idea, and then we started talking to some people. So there was at about the same time John, another Anthony John. He had released something like a notebook for Elixir as well, a very bare bones one. So we had some experience from Python. We brought him in, like, hey, if you're going to do this, how are you going to do it? What are the benefits? And then we're like, okay, so one of the things that we want to do is that we want to leverage the fact that it's very easy to build collaborative and interactive applications in Elixir. So it needs to be collaborative from day one. And it is. So there is a video on YouTube of me announcing LiveBook and it's really cool because it shows how LifeBook works. It shows Exxon as well. So there are some good examples. Hey, it needs to be collaborative from day one. And we really want to be interactive because one of the things. So for those who are not familiar with Elixir, the Elixir runtime, it's very easy to extract a lot of information from the runtime, like what your code is doing. We break our code into lightweight threads of execution so you can inspect each of them. So we said, okay, we want it to be interactive not only for people that are working with machine learning and numerical computing, but if you want to get data out of an Elixir system, like a production system, and try to say like, hey, where is my bottleneck? You should be able to do all that. You should be able to interact with a live system as well and interact with your neural network that is training. So this feature is not there yet, but it's part of our vision. And they said, well, what do people complain about in notebooks? That's always part of the research, right? So if you go to Jupyter, what People usually complain a lot.

What don't they complain about?

What we heard was like, well, the format that it writes to disk, it's not good to diff, it's not easy to version control. Right? So how are we going to solve that? The dependencies are not explicit and the evaluation Order is not clear as well. So how we can solve all those things. So we brought our set of inspirations, we bought the problems and we started working on how we want to solve this. And then a couple weeks ago we announced it. Maybe one or two weeks ago, we announced LiveBook, or maybe it was last week. Anyway, it's there. You can really see for our vision is not complete. You can see the important parts in there of like, it's fully reproducible. The evaluation order is clear. Your dependencies need to be explicitly listed so everybody who get a notebook knows exactly what they need to install and the notebook is going to install it for you. John he created a format called Live Markdown, which is a subset of Markdown that we use for the notebooks, which is really great because now if we change a notebook, we are just changing a markdown file, which means you can put it on virtual control. People can actually review your changes without having to spin an instance of that thing and make that work. So for us, it's a step again into this ecosystem. And I think there is a bunch of things that we want to explore and try out and really try to be a very modern approach to. For interactive and collaborative notebooks. And there are other things happening in space. So there is Jupyter notebooks. There's also Pluto JL coming from the Julia folks. There's also DeepNote, which is a software as a service. So we're kind of looking at everything and coming up with our own takes and ideas as well.

That's awesome. I'm glad that when you looked at this, you took that perspective of like, not we need notebooks. People love notebooks. But what's, what's wrong with them? Because I think there have been a lot of. There's notebook kernels for, you know, all sorts of different things for, for Jupyter, but they all suffer from similar issues. And of course I love Jupyter and it's powerful and people use it with great success. But I think after people have used it for so long, they've seen these consistent issues. I think, you know, the whole managing state thing that you mentioned and the execution flow is probably the top one on my list. So now you're really tempting me to try out. It also seems like you release something cool every week, right? I don't know how that works. I don't release something cool every week. So I'm feeling really deficient right now.

I'm with you.

I don't have anything new to release.

For now until next week.

Daniel, what you need to do is find some really talented universal students and get them to inspire them to work on some stuff for you.

I guess so, yeah.

Yeah. So yeah. So yeah. Jonathan has been excellent into this sense, you know, like, and it was like his first LiveView application. So I think it's both a testament to Jonathan and to liveview the fact that he could build this thing in three months while still studying, working part time and go check it out, go check the video. I think I'm really excited about LiveBook. It's really interesting. And so for example, we just merged auto completion. So when you're writing code there is now auto completion as you would get from VS code. We're using Monoco editor and everything's collaborative. If we have multiple working on it, the changes are broadcast. And based on this idea that it's built on Liveview where you don't have to write JavaScript, the whole thing, including all the Monocle extensions that we had to do. So it had the Elixir, Alexer and so on. It's like 2,000 lines of JavaScript. That's it for everything that it does to work. The whole thing about the notebook is that in my opinion, it was a very different approach to how we approach NX and Exon. It's like, hey, for NX and Exon, we're like, okay, let's build this and see where this lead us. But for Notebook it was like, this is an area that Elixir is really good at and I really want to have our own take on this. I think we can make this ours. Like our version of this, our vision, our understanding of this. And of course that requires looking around. But it was a very different thought process. Just like, hey, I think we can build this and I think we can build this great, because we have great tools for that. Just to make it clear out of the box. It works distributed as well. So for example, if you have a bunch of people using notebooks for some reason and you want to start like five machines in production and have people connect to those machines from anywhere they want, it just work out of the box. There's no need for external dependencies. You don't need to bring grad, there's no need to bring a database. So everything was really built using the. Again, if we go to the beginning of the talk, we're talking about the Erlang virtual machine and they're building telecommunication systems. Imagine you have this platform and you can build collaborative notebooks. So that was kind of our idea.

Our take how does it do that? Because it looks like it only runs on like local host. Maybe there's like a way to. How do you tell it? Hey, I've got 10 nodes that I want you to run across. Is that just configuring Phoenix?

So by default we run it on localhost. By default, if you're running our machine, you don't want to expose that and have somebody access the notebook.

Yeah, it's like an eval. It's like a public facing eval, right?

Yes.

Right.

Imagine that you are at a Linux request conf. Somebody would just be who is running notebooks here that I can't right now. I think we just need to tweak the configuration file. But one of the things that we are working on, we are going to get the release, we are going to ship both docker images and a command line executable. Then it will have flags for all this kind of stuff. Do these. And most likely what people want to do is that they want to say, hey, I am deploying this to Kubernetes. So I'm going to use something that uses the Kubernetes DNS manager to connect the nodes. So in Elixir you would use something like Purage or libcluster that figure out the topology that connects everything for you.

Yeah, and I can definitely confirm that people will want to spin these things up everywhere. Now I'm not surprised when I hear this, but the first time I started hearing production notebooks and I was like, how do you have a production notebook? It's a notebook, like how are you running a notebook in production? But this is like so pervasive. People are like, oh, this is my production notebook and this is my, you know, dev notebook and all of these things. I don't know if I go that far because I'm like, I don't know how to support a notebook in production. But it, it is such a pervasive idea. It's cool to see that as a piece of this. And of course there's other things too. Like you were mentioning, you know, pandas and other things. So for people that aren't familiar in Python there's a library called Pandas which deals with tabular data and you can do all sorts of cool like data munging stuff. So yeah, it's cool to hear you say that those things are on your mind and because you really use a cool thing every week, you know, maybe that will be next week or the following one.

Yeah.

Right now I think we are going to tackle Graphing graphs because it's part of the notebooks, but I'm hoping for the data frame stuff. Other people are going to step in and we are having a bunch of related discussions on the Arlane Ecosystem Foundation Machine learning working group and this kind of stuff if you want to talk about ensure machine learning. And then we can talk about neural networks. And there's so much work to be done and so many things to explore. So people that are excited, like jumping and you're going to have a feast. Right, because we didn't talk about, hey, clustering forests and classifiers aggressions and then we can talk about linear algebra. There is just so many things in the ecosystem that one can build and explore that there is a lot of work to do. And we hope people get more and more excited and they are going to join us in this journey.

Yeah, it seems like if you've got the graphing thing going and you're talking about Elixir having this sort of natural abilities with web development, with LiveBook and other things here, you know, a big thing in the AI world is monitoring your training runs with a bunch of cool graphs with, you know, something like a tensorboard or something like that. So it seems like, yeah, there's like, that would enable a lot of things. It'd be pretty sweet to have your, you know, your training run going in Axon. You kick it off from a live book and then you can pull up a, you know, interface to see all your nice training plots and all those things. And that's all happening in a really nice, unified, robust way.

Yeah, that's definitely something, you know, that we'll explore at some point. Probably tensorborne integration as well is something that we are bound to have.

Yeah, it seems like LiveBook really could be your marketing machine. You know, it could be like your way in for all the disillusioned notebook sharers out there who've had, you know, like Daniel said, they can do a lot of stuff with Jupyter notebooks or existing tooling, but there's pain points with collaboration with all these things. I mean, the fact that one of your headlines is sequential evaluation, to me that seems like, shouldn't that be how everything works? It says code cells run in a specific order, guaranteeing future users of the same liveness.

Not so quick, Jared.

I'm like, that's a feature how things work.

I mean, it's kind of the wonderful thing about Jupyter notebooks and the really hard thing about them, because it's similar. If you go back in history I don't know if either of you ever used Mathematica, but it's a similar idea. You have these cells of X execution. It's really wonderful for like experimentation, right? Because you can, oh, you did this. But when you're in experimentation, you expect things to fail almost all the time, right? So you don't expect to have like a script that runs and you unit test it and blah blah, blah. You expect to try something and fail and fail over and over and over until you like tweak it enough to where it works. And so that's great in the notebook environment, if you can tweak things like that. The problem is then like, oh, what were the 4 million things that I did tweak to get this to go? And what state is saved? Like in my notebook? Like I could get it to work and then reboot it and run it from top to bottom and it's not going to work again, Right? So it's, it's the good thing and the bad thing.

Yeah. And I'm pretty sure it's like this feature, let's say the sequential evaluation is going to be a limitation. At some point people will be like, hey, I started training my neural network, but now I want to do something else in the same notebook while the neural network is training. How can I do that? So we'll have to come up with ways of branching, but we want to be very explicit on the model. So we'll say, hey, you can branch here, or what we have been calling internal, because everything is organized in sections. We have to think, maybe I can set up some asides. So aside, they fork from a particular point, they branch from a particular point and execute the code based on those bindings. It's basically the state of the notebook from that moment on without ignoring the other side. So it's something we'll have to tackle. And if you look at issuestrecord, there are a bunch of things that we have been thinking about. So for example, one of the things that I wanna do. So we have the idea, so when you persist in notebooks, you're persisting to the file system. So on the issue is, for example, pluggable file systems. And I want to make GitHub a file system so you can easily persist your notebooks to GitHub and that works transparently from LiveBook without you having to say, hey, I need to clone and stuff like that. We can work directly on the repository. And I think that's going to be a boon for collaboration as well. Well, or not Collaboration. I mean, a different kind of collaboration. Right. You put on GitHub so somebody can fork and play with it.

I know there's like this thing in the Python world called Binder. So essentially you could create a GitHub repo with a notebook and then you click on the little badge and it just pops up a hosted version of that notebook that will run. So you can like give it a Docker image or something with all the dependencies. For someone like me, if there was like that tie in with GitHub and I could just launch a notebook and try like Axon, that's like. I feel like people would just latch onto that so quickly. Then the barrier is not like, oh, like Elixir's sort of new to me as a Python person, so I need to figure out the tool chain. But really what I want to do is I just want to like, click Shift, enter through a few cells and see how it works. And that's. That's very powerful.

Yeah, that's a very good point. Something for us to look into.

Yeah, well, you guys have done a lot, but there's a lot left to do. What's the best place to get involved? Like you said, fertile ground. Would you say hop in and have a feast or something? If you're interested in the space and in Elixir, sounds like there's lots of ways to get involved and to build out a lot of the stuff that's lacking. So is there a discourse forum or is there a Slack? Is there a community around this? Just you and the Dashboard folks working on it. What, what's the situation there?

Everything. We have the Elixir NX organization on GitHub, but a lot of discussion is happening in the Erlang Ecosystem Foundation. We have the machine learning working group. So if you go to the EEF website, you can get all the working groups there, you're going to find machine learning and then you can create an account, it's free, and then you can join the Slack and we'll be there. So that's where we are usually chatting things. Originally, a lot of the things, they were kept confidential, like live book, but now everything, at least everything that Dashbit was working on, it's out in the public. We don't have anything. No more secret projects. So that's the place to go and where we're talking about things. We have a monthly meeting where we meet and discuss and exchange ideas. So that's definitely the place.

Is NX bringing machine learning tools to Erlang or are there other Erlang but not elixir efforts in this space. You understand what I'm saying?

Yeah.

Is this the first time an Erlang, the beam based tooling around numerical computation is happening or is there like Erlang only things that have been going on?

I think it's the first time for the ecosystem and. Yeah. And because you can call Elixir from Erlang with no performance cost whatsoever.

Yeah, it's pretty cool, right?

You can just call like the numerical definitions, they don't work in Erlang because they, they translate the elixir ast. Not the Elixir ast, but they translate like the Elixir execution to the gpu. That wouldn't work with Erlang, but everything that we are building on top, like Exxon, because it's just building on top of the abstraction. So somebody could go get Exxon, call it from Erlang, build a neural network from Erlang and just run it and it should just work.

That's cool. Daniel, anything else from your side of the fence you want to ask Jose about before we let him go?

I'm just super excited about this. Hopefully there is some crossover from the Python world. It seems to me like the timing is such that people in the AI world very much are more open to trying, like things outside of the Python ecosystem than they once were. And so, yeah, that's my hope. And I definitely want to play around with this and appreciate your hard work on this and I'm excited to try it out and also share it with, with our Practical AI community.

Awesome. And I'm really glad that you are having me on the show and I was able to share all those ideas and this work that we have been doing.

Oh, you're welcome back anytime. All the links to all the things are in your show notes, so if you want to check out Jose's live book demo on YouTube, we got the link to that. We'll hook you up with a link to the Erlang Ecosystem foundation if you want to get involved. Of course, Axon and NX are linked up as well, so that's all. Thanks everybody for joining us and we'll talk to you next time. Thank you for listening to Practical AI. We appreciate your time and your attention. If you enjoyed this episode, help us out by spreading the word. Think of a friend, think of a colleague, somebody who would benefit from listening to it and send them a link. We'd really appreciate it. Practically Tactical AI is hosted by Chris Benson and Daniel Whitenack. It's produced by Jared Santo with music by Breakmaster Cylinder. Thanks again to our sponsors, Fastly, Linode and launchdarkly. That's our show. We hope you enjoyed it and we'll talk to you again next week. Sa.