Model inspection and interpretation at Seldon

Bandwidth for Changelog is provided by Fastly. Learn more@fastly.com we move fast and fix things here at Changelog because of Rollbar. Check them out@rollbar.com and we're hosted on Linode Cloud Servers. Head to linode.com changelog this episode is brought to you by DigitalOcean, the simplest cloud platform out there and we're excited to share. They now offer dedicated virtual droplets and unlike standard droplets which use shared virtual CPU threads, their two performance plans general purp CPU optimized They have dedicated virtual CPU threads. This translates to higher performance and increased consistency during CPU intensive processes. So if you have build boxes, cicd, video encoding, machine learning, ad serving, game servers, databases, batch processing, data mining, application servers or active front end web servers that need to be full duty CPU all day every day, then check out Digital Ocean's dedicated virtual CPU droplets. Pricing is very competitive starting at 40 bucks a month. Learn more and get started for free with a 100 of credit at.co changelog again.co changelog. Welcome to Practical AI, a weekly podcast about making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join the community and slack with us around various topics of the show@changelog.com community. Follow us on Twitter. We're @practicalai FM and now onto the show.

Welcome to Practical AI. I am Daniel Whitenack, a data scientist with SIL International, and I'm joined by my co host Chris Benson, who is a chief strategist for AI and High performance Computing at Lockheed Martin. Hey Chris, how you doing?

I'm doing fine. How's it going Daniel?

It's going good, no complaints.

It's.

It's been kind of gloomy weather here for a while in Indiana, but you know, such as this time of year I guess.

Sounds good. I'm about to head out of town for Live Works in Boston where I'm going to give a talk and it'll be over by the time this episode comes out. But right now I'm looking forward to heading off and doing that and meeting a bunch of people up there.

Awesome. Well, I'm super excited about our guests and our topic today. As our listeners know, we have a particular passion for the practical side of AI, hence the name of the podcast. We realized that, you know, a lot of times the blockers for AI projects are not necessarily the sophisticated modeling, but the whole productionizing things and operationalizing things. And one of the companies that's really leading the way in this area at Least I feel like is a company called Selden. And today we have Yanis Kleis, who's a data scientist at Selden, joining us. Welcome, Yannis.

Hi. I'm very pleased to be on the show.

Awesome. Well, we're really happy to have you. Maybe we could start things off by just hearing a little bit about your background and how you eventually got into AI and data science things.

Yeah, sure thing. So my background is actually in mathematical modeling. So I did my PhD at the University of Warwick here in the UK and there's a doctoral training center for complexity science, which is basically applied maths to real world systems and with a particular interest on sort of systems that are made up of simple rules, but that can result in interesting emergent behavior. So just to give you a couple of examples. So people in my cohort, cohort worked on various problems, sort of ranging from traffic modeling to modeling the spread of infectious disease. One of my papers was also roughly related to that kind of thing. So that's my background.

Awesome. And did that sort of academic training, did that transfer naturally into kind of industrial data science work? I know a lot of people, including myself, maybe had some kind of awkwardness or weirdness in kind of trying to transition from academic science into data science and industry. So it sounds like your work maybe was more applied though.

Yeah, I can definitely relate with what you're saying because the incentives in academia and in the industry are completely different. But the great thing about my department was that it was very multidisciplinary and people were working on all kinds of things. So I mentioned things like epidemiology research, but also there was a small group of people doing machine learning research with also some industry applications. So that this was a smaller portion of the center because we didn't have as many staff members doing research in machine learning. But. But there were some good ones. And towards the end of my PhD, I started taking more interest into machine learning and started going to paper clubs and be more interested in student talks about the topic. And as I was looking for a job in the industry and preferably a modeling job, I realized that this is the best way of going about it and I should really, really pick up the subject in my spare time if I can.

So, Yanis, I was wondering, as we were prepping for getting online and talking to you about this, is by chance the company named Seldon in any way related to the Harry Seldon figure in the foundation series?

Yeah, I was partially expecting this question, and that's exactly true. In fact, I'll tell you, before I joined Seldon, I had not read the foundation series, even though I'd heard about it. And soon after I joined, I asked this exact same question. And afterwards I read the book. So, yeah, thanks for the question.

No worries. And just for listeners, it had been a while since I'd read the series. It was a fantastic series, been around forever by Isaac Asimov. And so I was just. The idea there is that this figure, Harry Seldon, is what's called a psychohistorian, and it's essentially evaluating society based on mathematics. And so they're in a galactic empire and it's kind of. He predicts that it's about to fall and it's going to be 30,000 years of chaos. And he uses this psychohistory analysis to figure out a way to take certain actions to narrow the chaos down to a mere millennium. And so the series is about what he does and he creates these two foundations at each end of the galaxy filled with scientists and engine and stuff. But just since we made the reference, I wanted to relay that to anybody who had not read the book series.

Yeah, so it sounds like, I mean, kind of a central piece of that story is prediction, maybe kind of bridging that gap. I'm assuming that's kind of the reason why the Seldon name was chosen. But, you know, as you so kind of filling in the gaps of your story, Yannis, you came to Seldon. What is, what is Selden's relation to what they're trying to do in machine learning and AI and assuming it's related to that because of the name and the relation to prediction and all of that?

Yeah, sure. So the Seldon name is actually very fitting, as you say, because it's all about predicting the future. But what we are doing at Selden is we are doing machine learning deployment primarily. And this is everything that happens after your data scientists have finished their job and developed some models, trained them and achieved good enough performance so that they're ready to go out and be applied in the business. But there's a lot that needs to be done for that step to materialize. So typically a data science model, well, there's this whole setup of how you deal with the data and then the modeling script, which are typically Python or maybe R Script or maybe some other languages. And after the training is done, then you have maybe some artifacts, some model weights, which you can then load again and then make predictions. But that doesn't really make it easy for people inside the business to use the model. So what needs to be typically done is productionizing the model, which would in the simplest case involve wrapping it with some light API like a REST REST endpoint, for example, so that it can, it can start living in the company's infrastructure and other business apps can start communicating with it by, by sending requests, getting predictions back and then those predictions being acted on.

Could you extend that a bit and kind of talk about what the products that that Seldon has are and what projects it's engaged in and kind of give us a sense of what your customers or users are and why they're coming to you, what is it they're trying to solve when they engage in your products and projects?

Yes, of course, Selden is an open core business, meaning that our primary product, which the company is built on is open source. It's called Selden Core and it's a machine learning deployment platform that runs on top of Kubernetes. And it basically enables people to wrap up their trade models which can be trained using any framework. So you can be, your data scientists can work in Python or R or even Java or any other framework and then even Java. Well, yes, even if you feel like it. Yes. And because it's model agnostic, we don't get in the way of the data science modeling part. What we're interested in was once that data science part is done, you can wrap up your models into Docker images and then deploy them using Selden Core. And the models will be running on Kubernetes seamlessly and you can start using them in your business.

Yeah, and I mean, so I've kind of heard of Selden like a little bit in the past, but leading up to this conversation, I tinkered around with some more of what Seldon is doing and, and was super impressed. So much so that I'm encouraging one of the teams that I'm working with to use some of this for deployments. Because it was like really? So I had a model for reading comprehension and getting that into production was as easy as writing a particular Python class with a predict function and then, and then using some of Selden's tooling to basically just specify that, hey, I want a REST API on this port and my class is called this name. And then Selden kind of took care of the REST and that deployment piece. So it was super impressed with the whole workflow. And I'm sure I've missed. You can tell me what other pieces of Selden I missed. I'm sure I only touched on a little bit of what you, what you offer.

Yeah, and that's very great to hear that you're getting some value out of it already. So continuing on that. So Selden Core is the open source deployment platform and then on top of that we are building an enterprise layer which is supposed to make things that little bit easier and also more accessible to people that are not necessarily that familiar with the command line. So what we envision with the enterprise option is a centralized place to monitor all your models in deployment. So both monitoring them, sending off new deployments or decommissioning old ones, and having a rich interface of inspecting the models, how they're doing, and also have team collaboration and authentication of different levels of permission who's allowed to put new models in production, who's allowed to decommission the models, and that kind of stuff.

Cool. And I know that Chris and I have talked several times on this show, I think, and in our conversations outside of the show about this sort of weird friction that exists between engineering teams and data science or AI teams, often in that, you know, the tooling that the AI people are using is, is really weird in comparison to what engineering teams are used to like, you know, what are these jupyter things floating around? What. What do they do? And that creates a lot of friction, oftentimes in terms of actually building value out of the. The AI stuff. So in terms of who's kind of latching onto this tooling that you're building, do you see kind of people coming from both of those sides? So from like maybe a DevOps side and those trying to productionize what data science teams have passed off to them, but maybe there's people also from the AI side or what are you kind of seeing in terms of trends in that sense?

Yeah, that's a very interesting question, actually. And from what we've seen, both from community members reaching out on our Slack channel, but also in conversations with some of our clients, it is typically a very varied bunch. And I think it depends quite a lot on how data science, both the modeling part and the production part, is organized in companies. And there tends to be a difference, especially with bigger enterprises and smaller companies in that regard. So in bigger enterprises, typically places like banks, there appear to be quite a lot of silos between teams. So data scientists, for example, would only be responsible for the development of models. And once they're kind of happy, then they just chuck it over the wall to the DevOps people or the engineers who actually have to put it in production. And that's not always the best way of doing Things as the data scientists then don't get any feedback on how the models are doing and the data engineers just don't get feedback about, well, how you actually, what is this thing, how do you productionize it? Whereas in smaller companies it seems to be people are doing many roles at once. And this is something that when I joined at Selden last year, I also had to pick up. So instead of just doing pure data science and modeling, I had to basically had to take up the engineering best practices because we are at heart an engineering company. But yeah, the people are varied.

So Yanis, how did Selden actually get interested in doing model inspection and interpretability? What was the motivation that drove the company in that direction?

So to answer this question best, it's maybe fruitful to discuss a bit more about the capabilities of our open source deployment platform.

Absolutely. Sure.

So when people think about productionizing models, usually they think of single models. So you have a model, you wrap it up in a single Docker image and then you deploy it. So if it's a TensorFlow model, maybe you use TensorFlow serving for example, or if it's a Python model, maybe you write a flask app and then that's kind of it. It's a single model. You send requests of data through and get predictions back and then and you use them in whatever way you see fit. With Seldom Core, there's a lot more functionality that you can have. With Seldom Core we have this inference graph abstraction which is part of every deployment and it can be as simple as a single model, which is probably most use cases in the business. But you can do a lot more interesting things. For example, you can have several models running in parallel and then you might want to make predictions using all of them. And then before returning the prediction, you might want to combine all of them using some custom business logic, for example, well, it could be as easy as majority vote, or it could be just returning all predictions so you get ensembling at inference time for free. Another way you can use it is instead of ensembling models at prediction time, you can route traffic to models. So you could have say two models and have a router. It could be an a B test. For example, if you've developed one model and then put a second one model in alongside it, you want to split traffic 50, 50 and see which one performs best. Or you could have custom business logic, maybe you have several models and you know that one model does best at a particular time of day, for example, and another model does better at a different time of Day. So that's kind of domain expertise that can be coded in, into the routing or similarly if it's for example, a recommendation use case, maybe you have a couple of models and some of them do better on certain demographics and some of them do better on other demographics and then you can code that into your customer business logic router as well. So. So you can do quite a lot of interesting things with Seldon Core beyond single model serving.

Yeah, so in terms of the. So you just recently, you know, released and are promoting this Alibi project, which it's described as being concerned with model inspection and interpretability. So is the kind of, I've heard of interpretability in terms of kind of interpreting individual models, maybe sort of black box models that you try to gain some insight into. But then what is the difference when you're meaning kind of model inspection versus model interpretability, does inspection have to do with this kind of routing logic and where things act like which models actually got used for a particular prediction or what's the difference there? And the distinction?

Yeah, so that's a very good question. So when engineers talk about monitoring live systems, then they typically think of things like, for example, requests per second and the total load on the nodes that the models are living on and that kind of stuff. But when you're approaching the question of monitoring machine learning models from a data science perspective, then there are a lot of very interesting and useful things that you might want at inference time rather than just raw predictions. And this is what I sort of mean about monitoring and interpreting machine learning models. So just to give you a few examples. So one is we've already touched upon it and we'll go into detail is about model explanation. So you might want to go through your request logs historically and see, okay, well, why did this model make this particular prediction on this particular instance? And you could have a component in the Seldon core graph that is a model explainer, and then you can basically send that historic request through and get a model explanation back as to roughly the logic that it took at that particular time. Other use cases are you might be interested in whether your data distribution is evolving in time. So if it's changing, your model might become stale and then you might want to flag it for retraining or decommissioning. So you would use components that do outlier detection on the data or more generally concept drift on the data. So if your data distribution is constantly changing over time to create those alerts, and those can all be part of your deployment of your Selden Deployment.

So what does the tooling look like for these sorts of things? I mean, obviously it sounds like. I know we're about to get into Alibi and it's probably leading the way, but could you kind of describe the landscape of tooling and is it custom logic in many cases? Or how are people dealing with things if they're not using Alibi?

Yeah, that's actually quite a broad question. And in my mind, I always split these sorts of questions into two parts. There's the engineering part and then there's the data science part. And on the engineering side, it really is about, okay, what sort of components do we need that will talk to a model in production and how will that look inside, say, the seldom deployment inference graph? And maybe sometimes these components need to be stateful as well, which complicates things. And then on the data science part, it's less about the engineering of these components, but more about the algorithms. So what do you actually use if you want an outlier detection running alongside your model and detecting anomalous data instances? So that's really the data science piece. And we've done some work specifically on outlier detection for Selden Core, and we have examples on our GitHub about that. And with Alibi, we're sort of doing more, but more in the direction of model explanation.

Yeah. So maybe just describe as I'm thinking about, like. So I'm thinking about like Selden Core and how it gives you these deployments, which might be like one model or multiple models tied together with business logic. Let's say I'm already using Selden to do those sorts of things. So how does Alibi fit into that? Is it like a component that's kind of like a library that you call from like within those components? Is it kind of something that runs on the side and reaches out to those things and tells you certain things or analyzes logs? How does it kind of operate within this ecosystem?

Yeah, good question. So with Alibi specifically. So we got interested in model explanation as a company maybe around about four months ago, and we sort of thought, well, okay, it would be good to support model explanations for models for deployed models in our enterprise product. And then so on the data science team, we did a bit of research of what kind of techniques are out there. And in the academic literature there's actually a whole lot going on and there's a whole host of things that you could try to try and interrogate what your model is doing. But when it came to the open source code, it's Mostly research code, so it's not easy to use. So what we decided is that if we want to approach this seriously, then basically we need to recreate some of those algorithms. And so Alibi was born as an open source Python package for some of the more famous or well known algorithms for model explanation. So it's a completely standalone library, so you don't need to be interested in model deployment or any kind of productionization to try and use it. And you can use it in a Jupyter notebook. You can play around with some models that you've trained in the same notebook and see what kind of explanations your models can offer on the decisions that they made. But the way that it ties in together with Selden Core and eventually Selden Deploy, which is our enterprise layer, is that it will be the back end for producing these explanations of your models that are running in production.

The Data Engineering Podcast is a weekly deep dive on modern data management with the engineers and entrepreneurs who are shaping the industry. Go behind the scenes on the tools, techniques and difficulties of data engineering so you can learn and keep up with the knowledge to make you and your business successful.

Can you give a bit of an outline about the motivation for choosing Jupyter Notebooks in particular as the core interface for your data teams?

Yeah, and actually when I first joined Netflix, it was sort of tossed at me and I was definitely like, why are we crazy? And the answer was like, we might be a little crazy.

Go to dataengineeringpodcast.com to listen, subscribe and share it with your friends and colleagues.

So I noticed on the Alibi website you note that Alibi provides, and this is kind of the area air quotes here, consistent API for interpretable ML methods. And so you know what's in that API and is it an API that's used during training or testing or what other scenarios are you there for?

Yeah, well, API is maybe a bit of a grandiose term. It literally refers to the way we've structured these various kinds of explanation algorithms within the library. So it's, it's as simple as. So you could think of it in terms of the SciKit Learn API, which we all know, which typically has two steps for any given model or estimator. You have the fit step, which takes in a training set, and then you have the prediction step, so you can make predictions on new instances and with the model explanations, it's a reasonably similar process, actually. So, so some explanation methods require access to the training set to be useful. So you would have a similar fit step not always, but in some methods, and then really explanation time, explanation almost exactly maps to the predict steps. So you would have, rather than model predict, you would have an explainer explain and pass in a single instance that you want to be explained why the model made the prediction that it did.

Cool. Is this kind of model agnostic in the same way that Selden deployments are, in the sense that if you want to use TensorFlow or PyTorch or whatever, then these things kind of work out of the box? Or is this kind of restricted to particular reference implementations or that sort of thing?

Yeah, that's a very good question to ask. So in the first iteration of Alibi, we're now in version two, but we're still mostly focusing on what we call black box explanation methods. And when I say black box, I don't mean a complicated neural network that you've created. What I mean is that all you have is access to a predict function. And this can be very general. It can be literally something that just takes in arrays and spits out other arrays. So it can be as general as an API that's already sort of running in production. So we have a couple of methods in Alibi that all work on these sort of black box models. So it's very portable. We do have in our roadmap more model specific methods, because once you start to know a bit more about what your model actually is, then you have a lot more leeway and a lot more interesting methods as well that you can apply if you know the model architecture or the loss function.

So now that we've kind of talked a lot about what Alibi is in general, what the API is like, and how you integrate it into your workflow, I'd be interested, because we've talked about model interpretability in general on this show before, but I'd love to dive into a few more specifics because this is really a practical project that people can use. So one of the things that you talk about, so you talk about a lot of different methods, maybe one that we could start with is you talk about anchors and anchor explanation. Could you kind of describe what that is in general and when it might be useful?

Yeah, sure. And just before I dive into the anchor method in particular, I just want to pick apart a bit what we really want from model explanations. Right?

Yeah, that'd be great.

So there's sort of a couple of different notions of model explanation in particular, people talk about global explainability, where you typically want to know how the model performs, sort of, on average, say, on the whole whole of training set to try and draw some conclusions from it. And then there's the local explanation, which has been the focus of alibi in the first few releases, which is answering the question, given a specific instance and a model prediction, why did the model make this prediction? But as you can already tell, this question is not well defined. I mean you can't really answer it why the model made a prediction. You need to try and pick it apart a bit more. So one thing you can do is you can try and ask human interpretable questions what you would like about this model decision to be answered. And then given those questions, you can try and find algorithms that sort of approximate the answers to those kinds of questions. So for example, you might ask, okay, given this instance and this prediction, what is sort of a minimal subset of features and their values given which the model will make the same prediction regardless of everything else. So that's what we call an anchoring question. So that's where the anchor technique comes in. So you ask an interpreter interpretable question and then you go ahead and see, okay, how can I write an algorithm to do this?

So in that case with the anchor explanation is how would you kind of phrase these anchor questions, I guess, or what would be the thing that you would be looking to come out of alibi that would help you kind of with that explanation or make that explanation logically.

So the great thing about this is that these interpretable questions are asked when you want to design a new explanation method. But once the method is designed, you only need to be aware of what question that method is answering. So the anchors method would return say a small subset of features of the original instance and their values which would result in the same model prediction, say 95% of the time. So let me just give you a quick example, one example that we also have on the website. So it's the sort of semi famous census income data set from the early 90s composed in the US and it's a binary classification problem of predicting whether a given individual will be a low income individual or a high income individual. And the threshold is whether they make less than or more 50, more than $50,000 a year. And the various features of each individual are their education level, their occupation, their relationship status, but also their gender and race and various kinds of other features. So it's a fairly standard setup. You've got tabular data and you can train a binary classifier to solve this problem. So once you have your classifier trained, you can then pick individual instances and run The ANCHOR algorithm to try and find for each instance which really are the pertinent set of or subset of features that were important for this particular prediction. So, for example, I might pick, say, a woman who's mid-30s, who's been separated and maybe her profession is working in the government. And I run the model and it says, okay, it's a lower income individual. Then I run the Anchor explanation and what I get out is, okay, my anchor for this explanation is that their marital status is separated and their work category is government work. And with those two present, the model will make the same decision 95% of the time, regardless of, of all the other features. So that's where the name Anchor comes from. It anchor's decision to really what makes the most importance for this particular individual.

So wondering if you could kind of tell us a little bit about maybe a different method that you've implemented that might be useful in a different situation or possibly in the same situation in a different way. I don't know if that made a.

Lot of sense, but yeah, sure. So I can talk through another method or rather a couple of methods that we have in Alibi. So when we first kicked off Alibi, we sort of wanted to implement methods that didn't really have good implementations, but that were interesting and had received attention in the academic community. So Anchors is one such method. And actually it was, I'm not sure if, you know, it was designed by, by the same people behind Lime, which sort of kicked off the whole interpretability of machine learning area.

Yeah, I'm trying to remember. It's the one that kind of gives you linear estimations of certain relationships or something like that.

Yeah. So what LIME does, it kind of fits a surrogate model, a linear surrogate model around the instance. It's basically trying to approximate the, the nonlinear decision boundary with a linear one. And then the weights of that linear model can get interpreted as feature importances. So it's sort of a very familiar technique, but it has its shortcomings. And then the authors of Lime came up with Anchors, which they feel is sort of a better solution for this problem.

Cool. So we talked about the Anchor explanation. You list a few different methods. Like you were saying, there's multiple methods that are implemented in Alibi. There's the trust scores and counterfactual instances and other things. But I was wondering maybe as we kind of move towards the end of our conversation here, if you could just give us a little bit of an idea of where Alibi is heading and maybe also what, what seldon has in mind kind of for the future of Seldon Core and Alibi and maybe other things like where are you headed?

Yeah. So for Alibi we have a reasonably ambitious roadmap. I mean, I mentioned sort of our API mimics the scikit Learn API but for model explanation. So we kind of, in an ideal world we would be the scikit Learn of model explanations, bringing. Bringing various techniques together in one place for people to use and compare and contrast what makes sense for their use case. So other than that, so that would.

Be like scikit Learn. You can, you know, if you're wanting to do classification, you can choose from any of, you know, whatever it is. I actually don't know how many, like 20 or 30 or whatever implemented methods for class classification. So you're kind of imagining Alibi would kind of have that zoo of explainability methods that you could kind of call using a standardized API. Is that kind of the thought?

Yeah, that's kind of our ambition with Alibi eventually.

Cool, Cool. Yeah. What about Selden in general? Are there other things on the horizon that you can share or maybe just us things that you're excited about in the AI community or at Selden specifically?

Yeah. So I'm not sure if you're aware, but this week there's actually the COGEX event here in London, which is the sort of festival of AI where lots of companies sort of go and present their sort of emerging technology to facilitate all kinds of machine learning use cases in the industry. So what we're actually doing there this week is we're demoing sort of a first version of our enterprise product, seldom deploy, which has also anchor explanations running with a live request. So that's kind of quite exciting from a product development perspective.

So I guess as you are looking at that, I was curiosity with that particular event, is that more of a conference or is it more of kind of companies coming together to just demo their thing? What style of event is it? And is this kind of a. Are you essentially taking the opportunity to announce new product features in that with the one you just talked about?

Yeah, so it's kind of a bit of both. There are speakers, mostly industry speakers, but it's also. So companies have their own stands where they, they can demo their technology and that kind of stuff. So I expect there to be fewer people from the sort of pure machine learning research community, especially as it also coincides with ICML this week.

Cool. And I mean, first off, congrats on the announcement. That's really exciting. I can't wait to See what's next and the hype around that for sure. But. But let's say people are listening to the podcast they want to get hands on with Alibi and or Seldon Core and or Selden Enterprise. What are the best ways for people to kind of get up and running with Selden or Alibi? Where can they find resources and get help?

Yeah, probably the best resources is our docs. So you can just go on docs.seldom IO and that will take you to the relevant documentation either for Alibi or for Seldon Core, whichever takes your fancy. So it has a lot of information to get you up and running.

Cool. And do you have any. No worries if you don't. But as you've kind of explored this whole area of interpretability and explainability, do you have any recommendations maybe for people that are interested in learning more about that subject in general, as far as resources where they could learn about what has been done or what people are researching, what's the best way to find that sort of information?

Yeah. So one thing I can definitely recommend and something that we kind of got a lot out of when we were first developing Alibi is Christoph Molnau's book on interpretable machine learning. So as the title says, it's a book about interpretability techniques and it's, it's fully open source, it's available online and it's really well written. So if people want to get into the topic, then I think that's the best place to start by far.

Awesome. Well, thank you so much for joining us today. I know I'm excited to dig in more to Selden and to Alibi. Now I understand a lot more about kind of how it fits into my workflow. I encourage people if you have have questions about or thoughts about Alibi or Selden. Of course they have their Slack channel, but we also have our practical AI Slack channel which you can find@changelog.com community and our LinkedIn page as well if you just search for practical AI. So we'd love to hear your thoughts on machine learning, interpretability and inspection. Really appreciate, really appreciate you joining. Joining us, Yannis and hope to stay in contact and really looking forward to seeing what, what Selden does in the future.

Yes, thanks very much for having me on the show and I would just like to quickly thank my co workers Arno Van Merweren and Giovanni Vacanti, who both are working on the Alibi Alibi project and without whom I wouldn't have done the work.

Awesome. Appreciate that. And I really hope that we can run into you at a, at a conference sometime and, and, and catch up. Thank you so much for joining us.

Okay, great. Thank you.

All right, thank you for tuning into this episode of Practical AI. If you enjoyed this show, do us a favor. Go on itunes, give us a rating. Go in your podcast app and favorite it. If you are on Twitter or social network, share a link with a friend, whatever you got to do. Share the show with a friend if you enjoyed it and bandwidth for Changelog is provided. About fastly learn more@fastly.com and we catch our errors before our users do here at Changelog because of Rollbar. Check them out@robot.com changelog and we're hosted on Linode Cloud Servers. Head to linode.com changelog check them out. Support this show. This episode is hosted by Daniel Whitenack and Chris Benson. The music is by Breakmaster Cylinder and you can find more shows just like this@changelog.com when you go there. Pop Pop in your email address. Get our weekly email keeping you up to date with the news and podcasts for developers in your inbox every single week. Thanks for tuning in. We'll see you next week.