Clothing AI in a data fabric

As we look forward, we've been talking for years about kind of AI everywhere and IoT everywhere, Internet of Things everywhere and all of the things, and that there's a sense of scale in the future that we may not have had in the past that presents a fairly substantial logistical challenge to get all of that data from all of those different places to all the other places together and have a coordinated system that that provides the services. So data fabric, it's the trucking system of data. It's not the sexy part.

Hey, Jared, here. One of the things we can count on in the software industry is change. The state of the art changes so fast, in fact, that keeping up can feel like a whole other job on top of your actual job. That's why we created Changelog Weekly. It's our totally free newsletter that we drop in your inbox each and every Sunday. We link to the latest news, the best articles, and the most interesting projects that you should be aware of. We also added a little commentary from us saying why something's important, pointing you to other instances of a trend, or just making a dorky joke to keep it lively. So if you haven't yet, I recommend subscribing to Changelog Weekly and help us help you keep up with the Latest. Head to changelog.com weekly and sign up today. Again, it's totally free and we never spam you.

Yuck.

One last time. That's changelog.com weekly. Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive and accessible to everyone. This is where conversations around AI, machine learning and data science happen. Join us at PracticalAI FM community and follow the show on Twitter. We're RacticalAI FM. Thank you to our partners at Fastly for shipping our pods super fast all around the world. Check them out@fastly.com.

Welcome to another fully connected episode of the Practical AI Podcast. In these episodes, Chris and I keep you fully connected with everything that's happening in the AI community. We'll take some time to discuss some of the latest AI news and trends and we'll dig into a couple learning resources to help you level up your machine learning game. This is Daniel Whitenack. I'm a data scientist with SIL International and I'm joined as always by my co host Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris?

I am doing very well. It's kind of the end of winter as we're recording this. Starting to warm up a little bit. Got to say, my heart is with the People of Ukraine right now, because we're about two weeks into that event as we record this today, and spending a lot of my spirit cycles just thinking about those folks.

Yeah, definitely. Yes. For those listening at a later date, we are in the midst of a world crisis over in the Ukraine and war over there. So, yeah, it's definitely on a lot of people's, On a lot of people's minds. And I know we have, of course, Sal has partners all around the world and definitely, yeah, just really concerned and praying for those involved in really tough situations. So, yeah, it seems like in recent years, it's just like from one giant global craziness to. To the next, really.

It has been. We've had a string of them, it seems like with, you know, the pandemic being the giant global thing in the middle of it. But we've rolled right out into. Into this. And I didn't want to make the beginning of the show all about this because that's not what our folks are here to talk about, but just our hearts are with those folks. And when, when didn't we. I know we have some folks listening to us from that part of the world and we're thinking about you.

Yeah, hang in there. Yeah, hang in there. And it's not totally disconnected from the subject in that hopefully over the coming years, we can find more and more ways to apply technology and AI to really be helpful and beneficial in these situations. So not just used as a tool to give concentrated power to one party or another, but actually to democratize good things. I know in recent weeks we've had several different conversations about kind of geospatial data and satellite imagery and opening that up to people that could use it in multiple ways. And I think we talked about some of that in relation to kind of dealing with tropical storms or disasters, but of course, kind of wartime situations in the aftermath. It seems like a definite overlap with that general set of data and tasks is maybe, you know, I'm sure that there are people out there thinking about how we can, as maybe AI practitioners, work to provide, whether it's open data or open models to help in these sorts of situations. If you, if you have any ideas or, you know, people working in those areas, let us know. We'd love to have them on the show.

Absolutely. And getting technology out there into the hands of folks who really need it. And in a lot of cases, it doesn't yet. And that's a big problem for us all to solve.

Yeah.

And actually there may be some part of today's conversation can kind of help us along that path. There's a lot of context that this conversation could be had in to avoid conflict of interest. Because I'm in the defense industry, I'm not going to talk about it in the context of war and conflict. But we wanted to talk a little bit about some of the logistics of data and probably, I think the use case I've picked to start us off with at least is maybe kind of talking about health care. And I'm going to do this as a non expert in. And healthcare. And you too.

Neither of us are medical doctors.

Yeah. I wanted to talk a little bit about moving data around where it needs to be because that could be applied to just about any industry on the planet. And it's a need that everybody has. And there's a popular buzzword that goes with this these days, which is data fabric.

Yeah, it seems like there's just so many terms like data this, data. Like we've got data lake, data warehouse, data mart, data fab. I don't know what. There's probably other things out there like data bodega, data hot dog stand. I don't know what the. What the other things out there are.

Maybe we need a data dictionary.

But yeah, yeah, I mean, I know we're going to be talking about data fabric or like what people mean when they say that. Maybe before we go there, would it be useful to maybe just differentiate it from a couple of these other terms that people have in mind?

I think so. One thing I have discovered, just like in the early days we used to say this in the early episodes for those who have been with us for a long time, that many people define AI in many different ways and maybe that has coalesced a little bit around deep learning more recently. I think that there's more of an understanding about what we're talking about when we say AI. But just in the same way I'm discovering that data fabrics have many different definitions from many different people. So I'm going to offer my definition.

Cool. Add it to the mix.

Add it to the mix there and we can see. So for me, just like I did with AI way back when we were talking about that topic in one of our fully connected episodes, I define data fabric fairly narrowly and I try not to encompass all those other data star terms that we have pointed out exist. So if you think of a fabric being something that you lay down, like a physical fabric being something that you lay down, cross something across a small geography.

Curtain or blanket.

Curtain or blanket is a Perfect example. And it kind of covers those things and to some degree it connects those things. And so playing off that analogy, I would say that a data fabric is a way of getting data from data producers, points of origin to all of the consumers of that data. Some a destination, one or more destinations. And it may be across a very diverse digital landscape on how you do that, and with technologies of different types, different generations, completely different architectures. And a data fabric's purpose is to make the data go from point A to point B in a timely manner with performance guarantees and ensuring arrival, so that you can do productive things with that data at whatever the consumer of that data is trying to do with that data. So it's a logistical thing. Another analogy might be thinking about trains moving across the landscape or highway systems with large trucks moving goods and services all over the place, getting them where they need to be to all the. They may start at one place, but they may move those goods and services around. And so I think that is also a decent analogy for what a data fabric should do.

So I guess one question would be like, initially, when I got into data science, I had my first, you know, position as a data scientist working with data that was in a SQL database. To me, in that scenario, my sort of pattern of access to that data was SQL queries. Where does the data fabric sit in as a sort, as you sort of scale? Maybe all the, all the people that need access to data. Like, are we talking about just SQL queries or does this go beyond that? I imagine it does.

I think it, it goes way beyond that. It can be inclusive of that maybe. But if you think about historically with us, you know, all of us will running with that SQL example, we have a SQL database and we make queries to it and we're, we're pushing data into it and we're pulling data out of it and we're running analytics on it and we're transforming it. But it's all within a very, a very local context there. And I think as we look forward, we've been talking for years about kind of AI everywhere and IoT everywhere, Internet of things, everywhere and all of the things. And that there's a sense of scale in the future that we may not have had in the past. And so that presents a fairly substantial logistical challenge to get all of that data from all of those different places to all the other places together and have a coordinated system that provides the services. So a data fabric is really a big software system is what it is, a big distributed software system. And potentially globally distributed, depending on how you're. How you're using it. And so it's the trucking system of data. It's not the sexy part, it's not the thing we like to talk about in AI. But one thing, and I know this has come up in a whole bunch of conversations that we've had on this show, both with guests and just ourselves, is that there's a lot of underlying work to get data ready and to the right place and such, so that you can use it for your AI. And then you have the AI modeling process, which is a very small percentage of the overall process, and then it has to get deployed out and all this stuff. And a data fabric covers just the part of that, of getting the data to the right place so that you can do something presumably useful and productive with it.

Yeah, I was just having, over the past couple of weeks, having some discussions with another colleague at sal, talking about how we think about the products or the things that we build more in terms of, like, the outcomes we're trying to achieve with those things. So if I was to spin this a different way, like, how would I know if I had a data fabric? Like, what would the outcome be in terms of the way teams would operate or interact with that?

So, kind of thinking about characteristics that make up the data fabric to some degree.

Yeah. Like if I went into a new company and I was trying to figure out if they did or didn't have a data fabric yet, or maybe they were just stuck in the old days with, like, random, you know, other systems around. How would I determine, like, yes, they do or no, they don't?

I would say if they do, they probably have a specific software architecture, a system that is designed to move data around. And I think a notable thing about that system is it is highly scalable and is designed to handle n number of endpoints, N being any number that you choose to apply to that. And that the architecture fundamentally is able to support the addition and loss of endpoints as a standard operating procedure in it. Without it, that's not a remarkable thing. And you're not manually setting up connections to SQL servers constantly and that kind of thing. So you're able to take an endpoint, maybe an IoT device out there or some end user, maybe a tablet computer if you're in a hospital might be another case. And it just plugs into your data architecture and might work with many SQL databases and many other systems out there, aggregating the data in the right places and making it available for Use for there. And it's a software system designed to handle all of that complexity under the hood. Or maybe I'll say under the blanket or something, you know, here.

Nice. I ask all that, I guess just to try to. Because when you were first like, hey, let's, let's talk about data fabric, I was like, well, I want to know a little bit of what that means. So I kind of tried to learn a little bit. And the Google searching was not that helpful. So like I went to Gartner, which they always give a definition, right? And their definition, I'm not saying anything bad about the person who made this definition. It's probably really, you know, strategically worded or whatever. But it says a data fabric utilizes continuous analytics, a new term I didn't know either.

Continuous analytics.

Continuous analytics over existing discovery, governable and inferenced metadata assets to support the design, deployment and utilization of integrated and reusable data across all environments, including hybrid and multi cloud platforms. There's a lot of words there.

There's a lot of words there and I agree with many of the words. But one of the things I'll call out, it's my own bias, right? But I think I'm right obviously, is the I believe analytics is a separate thing. I believe analytics are. Systems are consumers of and producers for, you know, a larger environment where a data fabric may be serving that. But to me, that is a use case that a data fabric serves. And I have seen many people try to describe data fabrics in terms of analytics. And being the, the digital bully that I am, I tell them they're wrong, they're wrong. It's not that it's about the logistics.

I'm Jared Santo, GoTimes producer and a loyal listener of the show. This is the podcast for diverse discussions from around the community. The Go community. GoTime's panel hosts special guests like Kelsey Hightower.

And sometimes you can leverage a cloud provider and make margins on top. That's just good business.

But when we're at the helm making.

The decision, we're like, yo, forget good business.

I'm about to deploy Kafka to process 25 messages a year. It's nerd pride, right?

Picks the brains of the GO team at Google.

You don't get a good design by just grabbing features from other languages and gluing them together. Instead, we tried to build a coherent model for the language where all the pieces worked in concert.

Shares their expertise from years in the industry.

Don't expect to get it right from the start. You'll almost definitely get it wrong. You almost definitely have to go back and change some things. So yeah, I think it goes back to what Peter said at the start, which is just make your code, write your code in a way that is easy to change and then just don't be afraid to change it.

And has an absolute riot along the way.

Yeah.

You know that little small voice in.

Your head that tells you not to say things?

What is that? How do you get one?

You want one of those? Is it like an in App purchase?

It is Go time. Please select a recent episode, give it a listen and subscribe today. We'd love to have you with us.

So Chris, I think I'm getting what you're saying in that like in this definition or the definition we were talking about from Gartner analytics, sort of serves as a potential kind of plugin application to whatever we're calling data fabric. And then they describe this so it's. It's operating over and they're describing it at existing discoverable and inferenced metadata assets. That's a mouthful.

It is.

So metadata assets, I assume just means like data telling you what assets are available or the kind and number and type or like what's the, what's the metadata?

So I'm going to define metadata in the context of a data fabric. Meaning if you're talking about metadata in a different context, you might take it slightly different. But I would always define metadata in the context of the thing that you're discussing because it's meta relative to that.

I like how you're weaving these things together.

God. Oh, that was a bad one. That was even worse than mine. I hope folks are still with us. So I would think of metadata as being the metadata around the data, which are the messages you're moving around. So if you use the word messages to say I'm moving data from one physical place to another one, or one digital place to another one, and there are things that I care about regarding that data in the message, such as the security status of that data, you know, what you know. And that can be a lot of different things. The routing information of that data, who can access it. Yeah, exactly. All those things. The. There's a whole bunch of things where you could. That metadata, it could be in the form of tags, it could be addressing prioritization, it could be addressing, you know, recipients priorities. It may be that your data fabric is overwhelmed with the amount of data that moves through and you're. And you're having to make you have a service in your data fabric that is Having to prioritize those data packets based on your needs that you have informed in one mechanism or another the data fabric of. So there's all these metadata bits that are being attached to the data you're moving around. But having said that, what the data is the primary data that you're moving from a producer to the consumers you don't really care about in the data fabric other than the fact that you've attached those tags of characteristics and things that you care about that we just described, you're just moving it from one place to another and ensuring, providing guarantees that based on your priority and the importance and stuff like that that you've assigned that it's, it's fulfilling that, that promise.

So you mentioned sort of moving and like message, moving messages around and packets around and such, such operations like that. So let's, let's. If we bring it down to like the health care use case you were kind of proposing, let's say. Yeah, let's say I'm a data scientist and I'm creating a new model to parse healthcare records or something like that, right?

Yes.

And let's say there's an S3 bucket over here or something and it includes some healthcare records and there's metadata attached to it and it gets moved from like there to somewhere else. What's special about the moving part? Like if I have the metadata about like where it is and I know where it is and what's in it and who can access it, why is this sort of distributed or movement of data piece, why is that like a key piece of what you're kind of describing?

It may be that you don't really have a requirement for a data fabric if it's fairly basic. If you're basically saying there's one producer and there's one consumer and you don't have really an enormous number of them, you don't have thousands or millions of instances of a producer and a consumer.

So you're saying part of this would come in where let's say I had a thousand hospitals and they were all pushing records into let's say separate buckets or maybe even like a mix of on premises file storage, NFS or something and S3, is that more?

Yeah, I think the scale matters. There is a cost implementing a data fabric and that cost is not trivial. And therefore there is a level of requirement, there is a level of nature need that should be in place to substantiate paying that cost and recognizing that above a certain scale of operation the data fabric is saving you cost over the long haul versus producing more. So the point with the data fabric is that at some point the automation of all that becomes a cost effective solution. But if your needs are very low or minute in terms of what you need, you may not want to deploy a data fabric to cover that. If it's a very small operation, manually setting everything up might make more sense in a small operation. Whereas if you're dealing with millions or let's say thousands of data producers, well, it might be millions actually going forward. And you also might have millions of consumers of that data. That becomes untenable obviously to try to do all that in a very by the connection mechanism. So you need a system that can handle that at that massive parallel and concurrent scale.

So you know me, I'm always, I always try to get down to the distill, the practical. So if I get what you're saying, let's say we have the scenario where we have thousands of hospitals, all of those are producing medical records which in and of themselves have security issues and things attached to them. But then they're stored in, let's say S3 or mix of NFS or sort of a diversity of sources. So those are my producers. And then up at my consumers I have maybe a few different things. I have like hospital web apps or something for staff members to access records. And then I have like a team of data scientists internally that's building like models to maybe parse those records automatically in a better way. And then I have a third group of maybe researchers who are researching a certain disease or effect of a trial or something within a certain set of those records across different places. So in between those people. So I've got on the one side the data, on the other side I've got these different consumers sort of in the middle. If I put a fabric there, the things that would need to happen are each of those people would need to efficiently get the data that they need, regardless of what it's stored in. And they would need to be authenticated against that data and understand in a sort of like metadata way where all of the right pieces of data are located. So the data fabric would sort of do that in a metadata kind of protocol between the consumers and the data and make that process efficient. Am I sort of getting there?

I would argue you're getting there, but I'll actually, I want to throw in a couple of things that make it, that would raise the value proposition of applying a data fabric. You mentioned the fact that these, all these, these producers, the actual machines or whatever are going to S3 as an example. But what I would argue is the use cases that you described in the example, none of those are real time. None of those implied that there might be a life or death concern there. It might be that as we have more and more machines that are constantly producing data and sensors constantly producing data, that you could have critical care patients in that hospital, that it's not just an analytical thing. But there's also, but even the analytics themselves, there are analytics that are needed in near real time to save a life. And there are analytics that might be able to, to be happening, you know, kind of whenever, because you're doing a study maybe or something, and they're not urgent. And there might also be sensor data on that patient that needs to alert a condition that the, that the sensor is picking up. And you need a doctor in that room working on that patient immediately. So it's not going to go to an S3 bucket, or at least not just to an S3 bucket. It also needs to alert with specific data that goes right to whatever kind of unit that doctor is carrying around who's in a completely different part of the hospital. And that doctor understands I have an urgent situation and it's measured in seconds and I need to do something. And things like that happen in real life. It happens today in real life, but we don't have very good mechanisms to deal with that. And so if you can automate like that urgent thing to where you're able to prioritize that the data that's coming from a sensor that is critical immediately to a patient's life would get priority over a data that's just going to the S3 bucket at any point if there's any kind of resource contention. And so you're able to get the information to that. And it may be that they're not, not all things are going to the S3 bucket as a data store. There might be data stores that are collecting things for historical reasons, but you also may be sharing data directly between one device and another device without putting a store between them so that you get more efficient. Or you may be meshing multiple sensors together regarding that patient together so that you get a complete picture of that patient's condition at the time. And so as you do that, the data fabric gives you that logistical routing, prioritization, the metadata and the intelligent services wrapped around it, which may include AI, though you may have lots of deep learning models, but that are part of that mesh that are there to provide, that are able to actually do the metadata tagging, it looks at the data, it looks at other considerations that are inputs to the model. And the model is giving you inference that say this is what you do with the data right now. So the model itself can be a service and supporting the data fabric.

Yeah, so I guess in this case they referenced this inference metadata or discoverable and inferenced metadata. So if, if you've got sort of a producer of data, like you're saying a sensor or something like that, that's kind of under defined or annotated, something like that, then you know, maybe there's actually automation or models in place there to infer certain things about that, about that data and prioritize it accordingly or something like that.

I'm really glad you brought that up because if you think about it kind of going back to the very beginning of our conversation, there's lots of different types of technologies and different generations of technologies, modern, new stuff, old, antiquated, you know, legacy stuff that's out there and all of it needs to plug in. So another, another typical function or service of a data fabric is the connectivity to lots of different types of connected technology. In some cases it's essentially kind of wrapping a limited set of functionality and legacy so that you can basically bring more functionality to something that otherwise you would not think of having it and therefore make better use of it. So that's a key thing, is being able to actually connect all the things in a productive way.

Yeah. And maybe that connection piece gets to this integrated and reusable data side of things that that was in the definition. So you've got the metadata, but then it's sort of integrated and reusable across different environments. Whether that's the kind of edge environment like you're talking about with the doctor being alerted or something like that, or that's an environment where you're doing some sort of batch analysis for research purposes or.

Right.

Or something like that. And they specifically also call out the hybrid and multi cloud sort of thing. And I guess that that's probably because like in a healthcare hospital situation like we have in mind. But there are many other examples of this. Right. There's still a lot of mix between where data is stored at the edge or on premises or at a certain location.

Yes.

And the mix of where it's stored online. We'd like to think that everything we do is in the cloud, but it's not quite so it's not all.

And we have more and more things at the edge. And the edge is like this gigantic blanket Term, we talk about the edge. Like, oh, of course I know where the edge is. It's the edge, man. We know where that is. But the edge actually describes a huge variety of potential targets that you're trying to deploy data to or pull data from. And so, yeah, that diversity is important. And so we can address that in a moment.

So, Chris, you were talking about the kind of diversity of environments and platforms, and cloud means a lot of things and edge means a lot of things. I guess you've seen a lot of different types of edge devices and producers of data. And am I right that those could be. I mean, you've already mentioned sensors, but in a lot of cases. And I know I've, I've looked at a little bit of this for my wife's business too, like specifically in manufacturing too. A lot of times edge devices are actually fairly beefy compute nodes that are just, they're sitting in factory, they're not in the cloud or in a data center, but they're, you know, sitting out on the factory floor or something like that.

Yeah, endpoints at the edge can have huge capability. And not only that, but you want to be able to take advantage of it. And I think if you look, if we go back over this conversation and kind of, I'm going to tie it all together a little bit in a bow here is that the data fabric has a way of declaring context for different kind of data profiles that you're able to create to support whatever your need is that you're supporting there. And you might have a few profiles, or for at large scale, you might have thousands, potentially millions of profiles. Each of those profiles kind of this, this aggregation of routing, aggregating data producers, consumers that are all trying to affect a purpose. And so the whole idea here is to automate and scale this in a standardized way and allow the fact that you have endpoints that may come and go as doctors come and go from the hospitals, or machines are moved around, or it goes from one ward of the hospital to another, and that changes the capability in a particular area. All these things are possible by acknowledging simple things about where's my data originating, where is it going to, what's the metadata that makes it useful for the various purposes I have, and how do I capture that in a profile that's reusable over time in a productive way?

So I'm going to play devil's advocate here a little bit. Okay, so it seems like at a certain point when you had sort of disparate data systems and like if you wanted to do this, you sort of figured out where the database was and connected your app and like whatever that was all handled, at least in scenarios I've been involved with via documentation. Right, you have some documentation. And here's like how this system works and here's how this system works and et cetera. And you figured it out and you did your thing. It sounds like with this data fabric idea that you've sort of got all these. You still got all this diversity of producers of data and maybe the way it's stored, but then you've got this layer of metadata and tagging and rules which sort of brings all of that together. But in my mind, this metadata is going to end up looking like so complicated, right? Like, so how do you. How do you make it discoverable, I guess is what I'm getting at. So like if I'm. If I have a new application and I'm building the next best thing for my company and there's this layer of extremely complicated metadata, what advantage do I have in that scenario versus like the previous scenario where maybe I just go searching through docs to figure out which system to connect to or something to.

Abstract it a little bit. And because I think it's important maybe in this last segment that we do talk a little bit about possible implementation just to keep it practical, since we like to do that. If you're thinking of it that way, then you know, you have in endpoints that are coming and going. And so you have to have kind of a distributed registry of some sort. There's different options on how you might do that, but you need a registry that can handle that. And you also need kind of the equivalent, or maybe going back to our joke earlier about a data dictionary that kind of identifies the characteristics of the metadata and it constrains them to an operable sort. You can't have an infinite number of things or it becomes unmanageable. So in your use case, whatever your business is and your organization does, the data fabric has to be addressing the characteristics that matter to your situation. And you have to have a manifestation of that in the sense of these kind of dictionaries that capture those and registries of your endpoints with the right metadata to do it. So it's got to get real at the end of the day. Today, as we are sitting here in 2022, you usually do this through microservices with containers. I like to tell people, kubernetes in all the places and people were kind of coming into an era where people are not just thinking about a Kubernetes cluster, but they're now starting to think about multi clusters, sometimes maybe thousands of clusters that are connected. And you might put a cluster on a small device, you might have a whole Kubernetes cluster that is in something that fits in your pocket because of the things that it's doing and the way the services that you need there. So there's some really. We live in an amazing moment where not only is are things like data fabrics being ideated in terms of as a solution for massive scale that we're seeing coming into the real world, but also we're having these technologies that used to be hard and we're now getting good at them and we're starting to put them everywhere. And if you take advantage of kind of the way we do things across these different areas like Kubernetes and containerization, handling that layer, standard networking protocols underneath it, microservices to handle the things that the data fabric needs to do that are in a standardized way. It's a doable thing. You just got to constrain it. And you have to not try to, you know, to use the cliche to boil the ocean with it. It's got to be really practical to your use case and not trying to overreach. And that's why I tell people your analytics is not your data fabric. You're getting carried away. Add that in, you already have analytics. Plug it into the data fabric you build.

And so as an AI practitioner, if I'm sort of working with my organization to provide some better thinking around data management, maybe we're dealing with kind of distributed systems like you've talked about and a variety of producers of data and all of those things. As an AI practitioner and very often we might be part of these conversations, although maybe we're not like the architect of the system, but maybe part of the conversation. I guess some things that might be useful for the practitioner to bring to the table are, you know, sort of requests and advocacy for kind of standard protocols and access layers for data, which would include like prioritization and kind of speaking to like, hey, if I create a training data set, I want to make sure I. I'm not like overreaching. I'm using data that I should have access to. I don't want to get in trouble by using data I shouldn't have access to. But I also want to make sure that I can update my models. And so I want to make sure I have a standardized way of accessing this data that's coming from all of our devices everywhere. So that's maybe the AI practitioner consumer point of view, but also, I mean, an AI practitioner could bring things to the table. Sounds like in terms of the actual annotation and services built into whatever fabric this is, in terms of data's coming off of these devices, there may be ways to annotate them automatically with models.

Absolutely.

That make them more useful at the next level up, Right?

Yes.

So whether that's detecting entities in text or detecting anomalous conditions, which would change a priority of something, it sounds like that's a piece that the AI practitioner can play as well.

Oh, totally. And not only that, you actually didn't go where I thought you were.

Oh, okay. I never do, Chris.

No, you're addressing like using the inference to do the metadata and all that. But it may be that the actual data that's moving across the data fabric, not the data fabric itself, but the, the constituent data, that it's moving from producer to consumers. It may be that those many, many of those consumers may be mlops pipelines that are taking data in and doing continuous training. I had to say that. Or iterative training on a rapid scale and producing models or producing things that matter for. How do I treat this thing in the hospital? How do I treat that thing in the hospital? And it's going all the time. And none of that's the data fabric itself. Those are lots of AI models that are getting fed. We've spent several episodes talking about MLOPs. We kind of figured out how to do that. We've got some really amazing capability out there that our listeners now know about. And now if they can get the data from all those other places to where it needs to be, then you can go do all that and you can be that AI practitioner. That's incredibly productive because you're able to do it continuously. So it fits that piece of the puzzle where you need automation, standardization and scale.

Yeah. And it sounds like this is necessarily a multidisciplinary venture. So, like, I'm thinking, well, let's say that there's people out there in their own company. They're saying, hey, it would be great if we could move towards this layer of, you know, standardization and access, whatever that looks like in their situation. It sounds like that that is a multidisciplinary thing. So if you're like, including maybe AI analytics people, but also DevOps people or ML ops people and infrastructure people in order to kind of make this work, it sounds like you probably need some, like, buy in from those different groups and some willingness to. Because I could also see someone say, well, ah, this just seems like, it seems like a lot of work. I'm just gonna like, copy some stuff out to my own S3 bucket and like, deal with my own stuff.

We're in early days and noting like, you know, in the time that we've been doing, we've been doing this podcast for several years and in that time the resources available for deep learning have evolved quite a lot. And so we're in early days right now for data fabrics. And I think that probably not terribly long after we release this episode, we're going to discover they're just, you're having companies and open source projects and such, some are already out there and you're going to see a whole bunch more pop up because scale is becoming very real in this area and it needs, it needs a community of solutions around that to choose from. So I think you'll see a lot of out of the box data fabrics that can then be applied and customized. And there are some stuff that's already out there. Then we may talk to some of those in the days ahead. So it's just the infancy of this particular part of the ecosystem.

I'm glad to know what it is now and I feel more comfortable saying the term data fabric now, whereas before I probably would have cringed just a little bit to say it. So I appreciate that part of the conversation. It's always great to, you know, have these conversations and learn about a topic together in this format. Speaking of learning, we do always like to share some learning resources in these episodes where it's just Chris and I, and the one that I wanted to share, I don't think we have a ton this episode, but I think one I wanted to share, which I kind of forgot was open and online, is Jake Vander Plaas's Python Data Science Handbook. If you just search for Python Data Science handbook, there's a GitHub static page and you can see. So just to kind of give some of the table of contents, there's, you know, starting with IPython, moving into NumPy and data manipulation with Pandas, but then going to visual data visualization and machine learning. And each of these kind of has a way to run the examples straight from, from the page and open it in a colab notebook, which is always good to see and an easy way to run things and run a bunch of tutorials. So I would definitely, you know, take a look at this and if you want to buy the book, you can. But Also, there's more than enough to dig into on this online site and you can open it in, you know, the notebooks and such, too.

So that sounds like a really good one. And, and I'm going to offer one as well. The one that I'm going to offer, it's in the theme of data fabrics and the fact that we have containers with microservices and we're using containerized software to move things around. I like writing those microservices in go. I think it is a good language. If you're ready to move beyond Python into doing some software services and microservices around your data operations in your AI, then I think GO is a darn good language to do that in. So I'm going to, I'm going to reference go.dev.go.dev it is maintained by the Go programming language team. It has a lot of great resources to get going and understand the language, and it's a good place to start. And so if you have some pretty cool things that you're doing out there and you want to try to get it out there in the world by integrating it into software, this is an awfully good place to start.

Yeah. Awesome. Yeah. Well, I've been excited to get to chat about these things with you, Chris, and appreciate you setting me straight. So, yeah, looking forward to chatting about more in the future. We'll see you next time.

See you next time. Thanks a lot, Daniel.

All right, that is Practical AI for this week. If this is your first time listening, subscribe now at PracticalAI FM or just search for Practical AI in your favorite podcast app. We're in there. And if you're a longtime listener, please do share the show with your friends. It is the best way you can help Practical AI succeed. Thanks again to Fastly for shipping our shows super fast all around the world, to Breakmaster Cylinder for the Beats and to you for listening. We appreciate you. That's all for this week. We'll talk to you again next time.

Sa.