#149 Expanding the Scope of Generative AI in the Enterprise with Bal Heroor, CEO and Principal at Mactores

Hi, I'm Maggie, VP of Curriculum at datacamp and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead. Covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com there are so many different parameters which one model cannot sustain. So what we might end up with is multiple specialized models which are contributing to a final outcome. That is, we are quickly seeing that a paradigm shift in the generative AI model building is we might imagine each of the model contributing to an outcome is there are areas where we can standardize those and have different still generative AI large models generating outcomes, one for material, another for design, another for strength, et cetera. So those are very evolving topics. We are talking about really hyper cutting edge generative AI today.

Welcome to Data Framed. This is Richie. On this show we talk a lot about AI taking over everything. But for large enterprises it can be hard to make use of these new tools. From data privacy and security to picking the tools to use to the processes you need to safely adopt AI, there are lots of decisions to be made before you can get started. Helping you get these productivity boosts and cost savings while avoiding risks is is Bal Hiror, the CEO of Mac Torres. Bal is a serial entrepreneur and he's been building a platform to help enterprises adopt data and AI tools. I want to pick his brains on the challenges that large organizations face when adopting AI and the processes they need to put in place to get great results. Since he's at the intersection of AI and MLOps, I'm also curious to hear his advice on getting AI into production. Let's hear what he has to say. Hi Bal, thank you for joining me on the show.

Hey Hyruchi, thanks for inviting me.

I'd like to start off by talking a little bit about use cases. So one of the most common use cases that your customers have for generative AI?

Yeah, that's a very good question. So what is going on? Is generative AI right now is really helpful to generate a lot of text and images. So the use cases we are dealing since January are around marketing, generating images for marketing, generating product images using some base image and then using text generation to Generate a lot of content. And that content can be for digital marketing, for emails and content generation. But we see that slowly changing over time as there is more exposure to more models using hugging face the stability AI. There's so many different new models which are coming out. Claude is another great one which is changing the paradigm and it is helping us work with customers not in text generation, but now taking some customer data and generating some unique use cases of analysis. So we are working with customers to generate enterprise strategy based generative AI. So can generative AI generate strategies which will form some foundations based on the enterprise data and then help them quickly start building a business plan around that. So we are working with customers using generative AI to solve some enterprise problems. But that is a very recent last one and a half month progress. But for very long time generative AI has been wonderful in helping customers accelerate a lot of content generation. And we see that as these models evolve there will be lot many enterprise use cases as well as not many use cases with your own data. So these are some things which we are working on. One of the other areas which is getting interesting is creating this virtual environment using generative AI. Now let me tell you, this is very niche and very new for gaming customers. So as you know in gaming one of the major challenge is generating the scenes and generating the storylines and generating communications with the players is a major challenge. Generative AI is the perfect solution for that. And it is helping a lot of gaming companies create these virtual environments. Create graphics for this virtual environment and be very adaptive with the user preferences and auto tune the environment they like to work in or play with as well as discuss as a human with it or chat with the person as a human and not be very scripted for a very long time. Those were very scripted, didn't know the current topics current what is going on in the world. So now these agility AIs can be trained with all the news topics, all the trending topics which are upcoming and then in that arena and then help the gaming customers to really adapt that conversations, that images or that videos which needs to be integrated. Now at the video scale it is still far from giving the best outcome possible. But mostly for creating virtual environment which are most image based is getting pretty rapid adoption in the gaming community.

That's absolutely fascinating and I like that there are some very standard use cases like marketing and product that are applicable to almost every company. But then gaming have even higher requirements around AI and obviously been using AI for a long time. So they're taking it to the next level. Have you seen any other industries where generative AI is particularly in demand?

And this is very preliminary exercise we are working on is using generative AI in manufacturing. One of the major challenge in manufacturing, there are many facets in manufacturing. One of the major challenge in manufacturing is product design. Now manufacturing can be subdivided to discrete manufacturing silicon. There are so many different tentacles of manufacturing itself. So specifically we are working in discrete and silicon. And in those use cases, one of the biggest challenge is once you have a design, they want to see how those designs manifest in real time. So let us imagine this, right? So when you are designing a silicon wafer, you are actually taking sand and heating it up and cooling it down at different temperatures to create those layers. Imagine that like your circuit board, on the circuit board you have these copper wires going all around and then you are creating layers over layers over layers and compacting all those so you can have different circuits in each layer. Similarly, when you design a chip, you are trying to do that. Now lot of machine learning models and lot of technologies are today available. One of them is Cadence, which does a lot of great job. Then there are other players who do great job in using statistical and machine learning methods. But now what generative AI is doing is taking that data and actually generating a visual image of how those layers will form over time. Now let me remind you, this is one of the biggest challenge why the chips are expensive. If you go and buy Nvidia chips, if you go and buy intel chips, they are still expensive and they are not very, they are not cheap because there is lot of research which goes in how do I take this new design and stabilize it? And those are in half billions of dollars. It is like $500 million or a billion dollar research before you can release a chip for common public. Now they are passing on all of this expense to the consumer because they need to pay for the new chip. So now if this generative AI comes into picture, all those million hours of investment in designing can be substantially compressed. Because now generative AI can generate those layers and you can visually see them, which was impossible before, right? You have to actually manufacture it and see if how it looks like if those do not create any hotspots. Hotspots is like two layers touching each other which will create a short circuit. If the right amount of heat is not applied due to the design, they can have disconnections between the circuit. So there are many, many use cases like this in semiconductor, specifically in discrete manufacturing. So if you're building a specific product with AR VR, there have been some great enhancements in how do you take a design and then analyze it using say. Now with Apple VR it is going to be much, much different. But till now, with all these VR headsets, we could see the existing created 3D designs and analyze them as from a third perspective. But those were not generated by AI. Those were all generated by humans. Now what generative AI is actually trying to do is in that space you can use generative AI models to come out with various different possibilities and then you can select which possibility you want to go through without wasting a lot of time in design. So imagine you are building, let me give you a simple example. You're building a car. And now when you're building a car, you are designing the shape. If it is aerodynamic, what kind of look you want. Now a designer has to sit, think about it, build the whole car. And then in VR you can experience it before you start putting it in manufacturing. But with generative AI, what you can really do is you can generate the same car with various different materials, various different features, shapes, and then that can real time give you an interaction of how the final product might look like before you start working on feasibility of that product, market reach of that product, et cetera. And in manufacturing, we are exploring areas of generative UI which can be used in design phase, as in Silicon Valley, as in discrete. We envision it in the operation and manufacturing phase of a product as well. But that's still a very exploratory. But design is something which is very powerful, takes a lot of time, lot of effort, and provides immediate value for customers because now they can reduce the design time substantially with generative AI.

Okay, this is a fascinating use case that I've not heard of before and it sounds like this is potentially a huge market because if you can generate something as complex as a chip, then I imagine that it's also suitable for generating simpler objects as well. So is this appropriate for anyone who's designing products? Do you think you can make use of these tools?

The problem is going to be we will have to evolve different models. It cannot be one model like we used to think. One large language model like chat GPT can do a lot of things. It can write code, it can write emails, it can write bunch of content. Right. Based on the prompts you give, we envision that these models are going to be further now, specialized. As Sam Altman also said, the era of large language model is coming to an end, which makes less sense because one large model doing everything is good. For common use cases. But when you go with special edge use cases, you need models, generative AI models which are customized for a problem, for design. Designing a car needs to understand parameters of the material. It needs the strength parameters of material. It needs to know how that material folds and shapes in a particular environment and what kind of strength it will have. So there are so many different parameters which one model cannot sustain. So what we might end up with is multiple specialized models which are contributing to a final outcome. That is, we are quickly seeing that paradigm shift in the generative AI model building is we might imagine each of the model contributing to an outcome. It's still a very difficult thing to do because each of those models can affect the outcome differently. And today they don't communicate with each other. But there are areas where we can standardize those and have different still generative AI large models generating outcomes. One for material, another for design, another for strength, et cetera. So those are very evolving topics. We are talking about really hyper cutting edge generative AI today. So this is something which there is a lot of research going on, how to have separate specialized model communicate with each other. They can contribute to design. But yes, that will be a real strong use case. And like 3D printers, you might see people using generative AI to design models, then use 3D printers to print them and start using them in everyday life.

That's interesting that you think there's a shift from these sort of general purpose generative AI models to having more specialized models rather than a single broad model. Now some of the examples you mentioned, such as chip design and the gaming use case of creating scenes and so on, these seem like a quite, quite advanced uses of AI. And I know a lot of companies and organizations are just trying to figure out how do I get started using this. So do you have any examples of what might be a sort of good high impact first project or something easy that companies can get started with?

Absolutely. And that is what we do with called as Customer360. One of the major challenge today of products which are our companies which do retail so online and offline shopping. Right. So imagine retail products being sold in shops and then retail products show sold online. Say Nike. Nike sells shoes in the shops as well as they sell it online. One of their major challenge is understanding how they should market the product. To market the product, they need to know what their customers react to and respond to. For a very long time this was the done by market research and it was done manually with humans. And then there were tons and tons of people analyzing the data, taking the data from the retail stores, aligning it to the online stores, understanding the entire demographic, what gets sold where. And then there were solutions where people could align those identities and then create custom messages, custom offers in different regions so they can optimize their sales and optimize their supply chain so they can distribute the right kind of product in the right area and store their warehouses properly. Now this is all great, but now with generative AI what you can achieve is custom messaging for a custom target of people with the right kind of content which can influence a certain segment of your customers by taking the data from these large data lakes which customers have created in customer called as Customer360 in general. But largely what it does, it tells you different demographics, different responses, different reactions, using social media, using customer feedbacks, and it creates a strong foundation for these product marketers and designers to actually design something which is useful to you as a consumer. Generative AI steps in here today without any discuss, any custom models or anything. You don't need to do any fancy stuff out of the box. Generative AI models today can support generating custom content for them using your data. The only challenge is you need to host because one of the major Challenge with open APIs is the data cannot be owned. So when you host them you can partition the data from the foundation model. So in generative there is a very critical component called as foundation model. The foundation model is something which is trained on lot of data points and now it is ready for tuning for your data hyperparameters tuning, you can tune your model, make sure the data which you are giving is partitioned from the data which it already has and does not contribute back to the foundation model and then use that partition data transfer learning from deep learning. But now you can use your foundation model with your data plug, generate very customized content for your customers walking into door coming on your website and can be very useful for consumers like you and me. If I want to walk into an IKEA shoe and somebody can quickly tell me which is the shoe I would like and I walk out with the right of kind of shoe because of the content marketing, that would be a phenomenal experience. So I don't have to really go out there and search what is the right shoe for me, what kind of style I would like to, et cetera. So that kind of hyper personalization is now enabled using this use cases out of the box today. You don't have to wait for next two years.

Now you mentioned that there's a bit of A challenge with using things like the OpenAI API or in fact, I guess any commercial API in that you've got to avoid sending your commercial data to this because it's going to end up public. Are there any other problems or challenges that organizations might face when they're trying to adopt AI?

One of the major challenge we have seen in many organizations is that data is not of high quality. Now, generative AI is not going to solve and automatically solve the problem of data quality. And data quality can result in many serious problems. It can generate a heavy bias in your data. When generative AI is generating data at the same time, it can generate the totally nuisance data which is not useful. So a lot of companies who are serious about gen AI should work backwards, in my opinion, work backwards from the North Star, right? So if you want a great content to be generated, your input should be much better than which is available today. So how do we take care of data quality? How do we take care of making sure we follow ethical AI practices? How do we make sure we have scalable sources within our data ecosystem which can help customers make sure the data which is being sent to generative AI for training is of a quality which will generate better outcomes? Right. It's garbage in, garbage out. If you give garbage to generative AI, it will generate garbage. So it's very critical for companies to understand how to evaluate the quality. That means finding out is your data qualitative enough. The best part of generative AI doesn't need quantity. It's not like machine learning where you used to give terabytes and terabytes of data and then it would increase the accuracy levels or you might have to go and go back and find new features every time the data is changed. The benefit of neural networks and generative AI is you can give very small sample size of data and it can generate wonders. But if you give that small sample size of data with bad quality is not going to give you results. So the quality is very, very important. The second most critical aspect is governance. Now one of the areas we discussed about is, okay, now I want to generate custom content for custom content for Richie. So if I want to achieve that, if I give generative AI access of X everybody, it might end up disclosing information about me to you and your information to somebody else. So it may not directly do that, but there is a possibility because it's an end of the day. It's a mathematical model which is balancing out all the inputs from our personalized information and creating a math model. To respond to the next possible question you might ask it. So when it's generating a content, it is generating based on a probability of what you might like and we might both fall in the same demographical, possibly demographical and personality set where it would go and might disclose data. So we need to take care of data governance carefully and it's very hard subject in generative AI. That is because you need to literally partition the data in generative AI based on the data sets and so it does not cross populate the outcomes. So this is the second and major challenge and the third important challenge, which is more business challenge, is in the generative AI solution going to affect your business in a way which the investment results in the outcome. A lot of people today are saying opportunity where let's jump on it and everybody's jumping into it. So one, you have to take a step back and say hey, I'm going to spend, if not thousands, millions of dollars to generate this outcome. Because generating is not cheap. You need to get the CPU farms, you need to use a lot of infrastructure to train them. There's a lot of cost in research and cost in building these training data sets which can be trained on. So what we need to first start is first figure out what kind of business outcome my generative AI is going to generate for me. If we can hypothesize this Nike solution, if we start with an hypothesis, if this Nike solution can increase sales, increase their top line, increase supply chain solutions, reducing their operational costs. So there has to be either companies operate in two things, either top line or bottom line. Either you increase your top line or improve bottom line. And the third one is ESG sustainability. So if any of these three parameters have been affected by generative AI and what kind of quantitative unqualitative business outcome it is going to be, then you can work on a generative use case. Otherwise we have seen like in the era of big data, like in the era of machine learning, like in the era of deep neural networks, we have consistently seen customers investing millions of dollars and then the project or that that particular initiatives completely is destroyed because it is not generating business outcome. So it's very important not to jump under the gun of generative AI, but really look into the fact that do I have right quality, do I have right kind of governance? Is this business use case going to result in business outcomes which are benefit for benefiting business? Then yes. Then that's the step you should take. These are the three major areas we look into whenever we help customers with generative AI.

I mean there's a sort of old sad joke about how you wanted to be a data scientist, but actually you spend 80% of your day just cleaning data. And it sounds like that's not really going away with AI. You do need to really focus on that, make sure you got good quality data.

But hey, there are many great products or great solutions like Clean Lab. I don't know if you have heard of Clean Lab. It's a startup from Israel Teladiv and they have an AI based solution to clean your data. You don't have to train their models to clean their data. It's a very innovative self learning platform so you can check about that. And there are many other open source projects coming up where data cleaning itself is done by AI. As you said, 80% of the day time goes in cleaning the data. That might not be the case if you use the right tools. But the problem is not many people realize the data quality is an issue. You cannot ignore that. We have seen customers ignoring data quality and thinking, oh, I have a database, Oracle database of all of my data. I want to train, train that, train my generative AI model on my databases, which generally is not the case. And so I'm just trying to inside the thought process that there are tools available where you don't have to spend a lot of time in cleaning your data. But if you don't think about cleaning your data, it's not going to work.

You've also mentioned a lot of the problems with adopting AI around data privacy concerns and do you have any advice on how you can deal with that side of things?

Classically, people have been using users and roles. So you are a user, you have a role and then you have access to data which is not scalable enough in the AI world. So what we need to go and enhance is use attribute based accesses and policy based accesses. And I have written a lot of articles about it and have been actively working with our customers to help them understand when you do attribute based accesses, you are defining this data is say pii, personally identifiable data. This data is about healthcare, this data is about their location and their private data. So once we tag data and then allow our AI teams to access data based on tags, that becomes very scalable. One of the biggest challenge in data governance is management of governance. It's not about setting the right practices, it is about following those practices. If you create your systems and make it very hard to follow this practice, you might have built the whole China Wall, but you might have a back Door open where anybody can get in. What's the point? You spend billions of dollars building this wall of China, but then you have back doors getting into the country from anywhere. It doesn't make any, it's not helpful. And that happens because you spend a lot of time in following the process and the process itself was so tedious that it took a lot of time for somebody to make sure they have closed all the back doors. So that's why it is very important to innovate the way the data governance is done in this massive scale data where it's more attribute date, attribute based and policy based rather than very role and user based. So that gives flexibility to make sure every time the new data set has come I have tagged it. I don't have to care who has access into it because a policy is going to define who can access what data. Now when you are actually accessing data it will check. As a machine learning engineer, do you really have access to it? Can you train your models or these particular columns? It can be done at column level by the way, it's not only at table level, just hypothetically speaking. If you think everything in tables and columns. So you can attribute tables, you can attribute columns, you can attribute rows, so you can go granular, you can attribute files, you can attribute so many things. So attribute is a very flexible way of giving the right kind of information about that particular data and then the right kind of people can access it, which makes the machine learning team life easier. They don't have to wait for an admin to give them access. And if they don't wait for admin to access, there's no pressure on admins to give access to and then they will never make mistakes. So we are just taking out that unnecessary grunt work of admins to every time a machine learning team or a data scientist asks for an access, they have to go through a whole workflow. The data is coming in, it is pre tagged, policy says you can access it or you cannot access it. You are not defining based on my request every time. And they have to find which data set and give access to it. So we are just changing that thought process will enhance lot of governance best practices. Other than that sir, I'm talking a little bit more on this subject because this is very critical. Other than that one of the most important aspect is explainability of the AI and there has been so many, so much of research going on that that how do these models work? Why are they taking out? Why are they providing the outcomes, what they are right so actually it's called explainability AI or explaining the model. There are many open source as well as proprietary solutions available on aws. Something that's clarify and there are other open source tools which can help you explain why your model is behaving the certain way. And when you find that out, you have to constantly monitor those models to understand why it is taking decisions the way it is taking. And once you can explain that, you can immediately understand what data is trained on and also identify if there was a mismanagement of governance. So it's not only about protecting with your walls, but also monitoring nobody's getting through those walls. So it is very important that. And how do you monitor? You monitor it by counting the population every 10 years. Correct. And the same similarly you cannot do this for every 10 years machine learning. You can actually monitor the model and understand how what data it is getting trained on. So now you know there is no governance challenge. So these are the two levers you can pull to make sure you are following good ethical AI practices.

That's really interesting that you mentioned data access management. And also I guess the other one was explainable AI as being like the. The two things that you really need to take care of. And actually on that first point I think one of the big challenges, particularly in large organizations is tracking the lineage of data. Like what where has your data come from? And so do you see similar challenges with tracking the lineage of AI?

A lot. So what has happened over time is lineage of AI was a very, very important topic in data warehousing. So I'm talking about say six years before or five years before data warehousing. Before big data was a big thing. It used to build data warehouses. It was very tabular and structured, not unstructured data sets. And it died over the last five years. And all these products like snowflake databricks and these products have taken over which allows you to do a lot of fancy stuff. And people have started slowly ignoring the fact of which this data warehouse world built over 15 years is to have quality of data, lineage of data and having a strong governance. Those three tenants somehow been blurred over last couple of years because there has been hyper productization of these big data solutions as lake houses or as data warehouses. Essentially they took the concept of big data and implement it as a data warehouse. But lot of ingestion practices of those data warehouses are not carry forwarded from those old school techniques. So informatica talent. All of these guys had a plethora of services to do the data quality but they themselves are not scaled enough to support this massive ingestion of these data sets. So people are just now dumping that data into data lakes and data warehouses without realizing these three tenants are very critical. Lineage is a big issue in machine learning. Everybody we have worked with and when we have implemented explainability of data, how did this data come in? Oh, that was somewhere in my data lake. And how did that data lake build that data set? Oh, some spark job ran it. How did that spark job get that data? Nobody knows. So lineage is a major, major challenge and it is a very confusing topic for various people because lineage can mean two things. One is a lineage which you maintain in data catalogs where you show how those tables are getting evolved from source tables. But another lineage is how do you track which file was used to process. Now with data warehousing systems it was easier. You can just monitor the table transformations and then generate the final lineage cycle. But with this advancement in spark and using all these tools which we use DBT material, there are millions of those. Now, we generally process data from a file and nobody's looking into, okay, what was the file which was sent to me from where did it come? What was the ingestion point? If it is real time data stream, then which was the session which was used for the real data stream? How did that data come in? And if I'm training on something, is it that is data been certified? I'm not saying mastering of data is required, but at least you need to know if the data is coming from a certified source which you are now using to train. So lineage starts from the ingestion point till you reach it to the consumption point and it has to track everything. I have not seen a solution which does that today. End to end. There's no solution which tells me file, version, file number, source, everything from start to end. We build those systems. It's a build solution. There is no out of the box product which does that because of the ecosystem, right? So you have various different points, you have various different processing points, and you have various different consumption points. So this cannot be an out of the box product. It's a too complex problem to solve. Let me give you a good news. There is a generative AI way of looking at it. Since you are on the topic because of all the logs. Every tool set create so much of logging, of course, if you enable them. But that logs can be very powerful in answering lineage for some of your rows. And you can use generative AI models to track them and create reports of lineage through generative AI. This is where we use generative AI to manage data ops. So DataOps is one of the subject we're talking about right now. There are many implementations of generative AI within data quality, within data lineage, within data governance we can explore and that's what we are recently been working on. How do I make somebody's life simple using generative AI in managing all of this mess of data ops?

It does seem like there's then a trade off between having very large data sets where data governance becomes more difficult and DataOps becomes more difficult and maybe working with a smaller data set where it's easier to manage this even with more traditional tools. Moving on from this, I'd like to talk a little bit about like the roles and responsibilities within the organization for actually getting AI into the company. So who in an organization needs to be and responsible for AI?

Responsible should be the CEO of the company. Yeah, so it's a good question. The various organizations have different kind of structures and we are a very strong proponent of business driven AI decisions. What does that AI strategy, what does that mean? Is the business owner of a particular division or a particular product should be responsible to defining the outcomes. Now you need a team of people to execute it and that can come from it. But if the IT is not solving business problems, then it's not really doing anything. So let me explain you a little bit further. For very long time it was treated as an independent department on its own. And their job was to create infrastructure so people can do their work. And that was not a business outcome. For a very long time it was treated as operations. You get your desk, you get your monitor, you get your laptop, you get lights, electricity and you can work. It doesn't matter. It does not affect my business. But with AI, with generative AI, with data analytics, the world has changed. It is no more just a facilitator. It needs to be aligned with the business strategy. And that is what we are striving and helping our customers with that not to treat it as an operation, as a cost center and treat it like people who will provide me infrastructure whenever I need. Rather treat it like somebody a part to partner with. So your business ideas can translate into more technology backed solution which can change your business substantially. So the roles which we see being successful is a business leader of that unit leading as a decision maker of how a particular AI solution should manifest. Then you need people from technology to really think about how this interworking of these structures are going to work. And then you need business validating the outcomes of these people, of these structures, giving out right kind of business results and monitor if they need improvement in efficiency, improvement in cost, etc. So a VP of a business should have a team of data architects, data analysts and machine learning and data scientists, which can very well be an IT function, but reporting to him. And then they need business analysts and business strategists who will look the outcome of these folks who have designed these pipelines and show that and validate that the results which they're generating is affecting business. And so you can say, Okay, I invested $5 million in my technology and is generating $50 million. If the outcome is not 10x or 5x based on every company, what's the point? What's the point of investing in technology? So running this as product. So that is where we came out with the idea of data products rather than data platforms. It generally goes out and say, we want to get a data platform, let's get databricks, let's get Snowflake. It's no more a platform solution. Now today it's about data products. How do you build data products within your organization which can then supercharge your business teams to go out there and generate more business? And that is the philosophy we should be getting into rather than treating everything like a data platform.

Okay, so it seems like there's a lot of people involved in here. So you mentioned, like you have some executives, maybe the CEO getting involved in this from the top. You've got business people involved, you've got technical people, you've probably got some IT people involved.

By the way, that was a joke about the CEO.

Oh yeah, yeah. Well, I guess legal teams are maybe the only people I don't think you mentioned. But does there need to be some sort of legal input into any AI project?

Absolutely. And when I said business, I, I somehow automatically thought like legal. It might be, I might have missed saying legal. So business analysts and strategists generally every time loop in legal teams and that the legal teams are, are part of that whole decision making because there is a substantial amount of change. A, there is a lot of IB B, there is a lot of areas these AI products can affect the customer's business in a way which will need change in contracts, terms of services or privacy policies. So legal has to be part of the discussion before you release it to general public. But legal slows everything down. So our general philosophy is building a product called as mvp, Minimum Viable Product, a data product internally without involving legal. And then once you have a minimum Viable product, which is not affecting because it's not providing any, it's not structurally changing anything for your consumers or customers. Show that to legal and see how the contracting and the legal framework needs to change. That has helped in a very positive way because when we did this for a customer and then went to legal, Legal said, now we can actually see what you're building. If you, if we go to legal very early, one of the challenge which happens, they cannot imagine what is. Their whole mind is based on the legal framework. And then they will slow down the project, slow down the implementation substantially because they want to make sure you are legally following all the boundaries. So legal is very important, but when you go to them is very important as well. So once you have a working prototype, working prototype, you know what it is going to do. You showcase that to legal, you showcase that to marketing, you showcase that to the sales team. And once you showcase that, everybody are on the same page now, okay, this is what this is going to do. And then they can use their sales and marketing techniques to now change their messaging. Legal can go and change the legal framework to change the message. So that's why this business strategist and business solution architecture is required is because they are going to now manifest all this technology mesh which has been created by the technology team to the legal teams and sales teams and marketing. That's why we said it should be product and it should be very business focused rather than very technology platform focused.

That is interesting that you mentioned the idea of timing. So if you speak to the legal people too soon, they'll tell you no and shut things down. You wait until your company's getting sued, price probably a little bit too late.

Yeah, that will be super late. Yeah.

Because things are moving so fast in AI at the moment. How do you balance speed of adoption with making sure that things are done right?

Very good and interesting question. There is no right answer for this unfortunately because AI is like a Cambrian explosion. So you do a lot of things and few of them will be successful. It's like a startup world today. Unfortunately, we are not in an era of AI which would be fit it and forget it. That's why it's called AI experiments. You need to have a very scientific view of AI rather than having a mathematical view of an AI. And this is an analogy which I want to give coming from science. I'm a huge physics guy, so I like to follow physics, cosmology, learning the scientific method rather than Socratic method. Socratic method is where you communicate with other person and try to explain them if it is right or wrong. And a scientific method is you're observing something and make sure your hypothesis are validated. And if they are validated then why they are not validated. If they're validated, how they are validated and you keep on learning from failures and not argue with somebody why it is right. It's a fact based discussion. Whereas software method is more philosophy based. So what is going on in the market today is a lot of technology teams and businesses are very socratean in method, right? I know this, I have been doing for this for 15 years or 20 years. And this will work without data, without really experimenting on things. And AI is exactly opposite of that. In AI you can never tell how things will go. You have to experiment, you have to see how it will work, you have to form hypothesis and work from that. So that is this two world problem which is being, which is which people need to slowly solve. Now when you have these two different worlds which you are dealing with, the scientific method wants you to stop, scientific method wants you to observe, scientific method wants you to adopt changes after you have seen the experiment work. Whereas the socrated method is very much in gut feeling that okay, now this work, let's go, go, go, let's do new thing, right? So that's where I came. This two old philosophy is you need to maintain the right balance between what decisions are scientific, what decisions are Socrates. So something like I have built a product, it is working, it is showing outcomes, it has been validated by the scientific method. Now how the adoption works needs to be accelerated. But once you get the feedback, you have to throw the ball back to the scientific method and say I have got this feedback. Does this make sense? So it's a very cultural problem. So because adoption will never should not drive implementations, adoption should drive research, adoption should drive understanding why the adoption has increased rather than just coming out with new features. And this can be phenomenally explained by Apple. So and why am I taking this little differently is because Apple is the one company who never releases features as a response to Android or any other OS maker. There has been, they have been critiqued for so long that hey, my Android has 500 new features every year and Apple has not released them for last three years. Apple never releases any of these features for years. What they do is every time they release a feature they take feedback to us, take a scientific approach. Is this needed? How much people are going to be affected by this? I don't, I never work for Apple, but I'm just imagining what they might be doing. How much efforts are needed to build this feature. Is this going to really demographically change the adoption of the product? And that's why you see we are coming after five years after Oculus, after Meta took over almost five years, Meta was going behind. Oh, let's get VR, let's build this Metaverse and build all of these fancy stuff. But Apple said no, we want to take simback, see the adoption, see the research. So Apple is a good example to show. And now they have made some horrible mistakes in the past. I'm not saying they have never done, but that's a very good place to start learning how the adoption and implementation should change hands when you have feedback.

I think it may be even longer ago since Meta or Facebook at the time acquired Oculus. Like there's a good sort of, what is it? 2014, something like that. It's going back away. So yeah, you're right, Apple's a little bit behind the curve there. But yeah, they're taking the time. So you mentioned experimentation. And just on that note, I think that's one of the areas where data teams are going to get involved in AI. And can you maybe talk more generally about data teams? So all the data practitioners, the data scientists, data analysts need to be thinking about in order to make use of AI.

There are various things, right? So first of all, today start adopting AI driven code writing which is available today, use GitHub, Copilot, AWS, CodeWhisperer. There are so many different products. Stop waiting to write the code manually. You know, there are so many tools, we use it every day and it accelerates your code writing skills. So that is the first. There are still I, we, we meet with a lot of people who are not using AI to write code. But other than that, what we have seen, or I have personally seen, challenge with the data teams is looking at a big picture and then knowing what is your role in that big picture. One of the major challenge which happens in most of the data teams is these individual engineers, machine learning teams, AI experts, data scientists, even data scientists have a very, very myopic view of very small scale view of what is my role, but do not have the larger picture of what is going to happen. And the top decision makers have just the larger picture and they never see what is going to work at the ground level. And it's very hard for an executive to go deeper because there are many other responsibilities. But at some point in time it is easier for a engineer to look at the Entire platform may not understand every component. You don't have to, but you at least know that my robot is going to be part of building this entire megastructure. Right? Imagine every of the components these guys are building are small robots which are again building a larger robot. Right? If you believe in that philosophy of using small units of building to build other bigger things. So if these people are going to contribute to something, they need to know how it is going to be manifested, even if they don't understand every component of it. Because that gives clarity, that gives a very strong sense of path to take, right decisions. We have seen most companies, products or projects failing because there is no clarity across all the team about the small scale implementation or the smallest detail and knowing the largest aspect. So change in the psychology philosophy of the data teams is very important. Second is retooling and reskinning every six months. One of the major challenges I've seen, oh, I have used Spark 3.0. It has not changed since then. No, it has changed 15 times after that. Right. Then there are so many new tools available which can save my time. Not only save my time, give me better results, adopt them, learn them, Invest in learning. Invest in learning. Every three months, I personally spend at least four hours to say eight hours off my week based on the week finding out what, what am I doing which is not up to the standards of others. Right? What am I doing in my general everyday activity as well as my learning about technology, which is not cutting edge. So you don't have to be cutting edge, but you at least need to understand that your job is paying you. But as soon as they stop paying you, you need to find another job where the amount of skills you'll require will be exponentially high. So not stopping to learn is the philosophy you need to build. Never stop learning. So that is another very important piece I have seen. In fact, in our organization we have compensated 20% of their work time to be associated with skilling. So without Skilling, there is no progress. And the third and most, you know, most interesting aspect is not being overwhelmed by technology. Have seen a lot of people getting all, oh, generative AI is here. All our jobs will be lost. No, that is never going to happen. But running behind the crowd all the time. I am a spark engineer. For example, I'm a spark ETL engineer and there is generative AI out there. I should not drop all the ball and start looking at the generative AI. Start looking into as a spark engineer, how can I influence generative AI? How can I use my skills, my super skills to be contributor to the evolving scene outside. So how to think as a contributor rather than running, being in a rat race. So as a spark engineer, I can say I have built a phenomenal framework which can help you do lot of transformations in a way, which is very helpful for generative AI. Like generative AI cannot understand tables. There are some, but they cannot. So I can write a spark job which can convert my tables into long forms of text which say Claude can be trained on. I build that and go to my boss and say, hey, this is what I've done. I know I don't know generative AI, but I know what input it needs. So think differently. Think from a perspective of how can I contribute to this evolving landscape rather than jumping onto the ship and then figuring out, oh, this is, this is either way beyond me or it takes too much time or wasting a lot of time to reach your end goal. So I generally, you know, advise people on making sure you can contribute and skill upskill yourself.

That's brilliant advice. And I particularly like the idea that you need to be continuously learning. You need to continuously upskill yourself. So on that note, do you have any ideas for particular skills that data analysts or data scientists or other data practitioners might want to learn in order to get into working with AI?

The first thing I try to tell anybody I meet is first go and learn foundations. So what is data? What is data structure? What are the foundations of data? Lot many people do not understand foundations of data. So there is a tabular data, structured data, unstructured data. Right, but what is the form of unstructured data? Unstructured data can be key and values. Unstructured data can be graph in nature, unstructured data can be columnar in nature. There are so many ways unstructured data can be and then there is still structured, but we call it somehow unstructured, say HTML. HTML is still structured. It is in a structure, but you have to read it differently. Then you have unstructured data. There are so many philosophy, they're not philosophies. I would say there are many techniques in unstructured data, right? So you have pulsar architecture, you have star schema, you have so many different aspects. Learn from the basics first, like understand why this was evolved, understand the history since 1960s, how the data got evolved. So you are part of that history now, right? And you can know that in the era you are living, you are sitting on the shoulders of giants who have built this over years. And now it gives You a very strong perspective that where it might lead, where now the next innovation might happen. So start with the basic foundations. But now once you have done that, what you should be really skilling yourself is to understand these new languages which are coming out. So people are trying to use Rust in analytics. People are trying to go in analytics. So Python has been the de facto language. Python R has been de facto language because of the functional nature of it. So I would advise a lot of data engineers to be flexible in their language skills and understand various various languages which are possible because each of them bring different benefits to you. And then the third one is learning new tools as they are coming out. So when the new tools are out there, I cannot name them because I don't want to be very salesy here. But there are so many new tools in data analytics which are out there which allow you to quickly do your work. For an example, Trifecta is a great tool. You don't have to really wait and build a pipeline. You can use Trifecta to generate a lot of spark. There are other tools similar to that which can be used. So try and expand your skill sets from the tools perspective. So a combination of tool and programming is what your job should be rather than just programming and not thinking about any other, you know, any other tools out there or just using tools and not thinking about program. So the world the data analysts, data scientists should be living in, what can I do, where I need to write code and what can I use which is already there? So mix of both will help them a lot.

And I have to say a lot of your answer there was really surprising to me. Like I didn't expect you to talk about things like data modeling or using Rust and go and languages like that. So I suppose challenge for all of our listeners then if you are using Rust or go to work with data, then please do message me on social media. I'm Richie Rocks most places on the Internet. Let me know what you're using it for there. So yeah, that's a really fascinating take there. So just before we wrap up then, can you tell me about what exciting things you're working on at the moment?

So as we started with the call, we are actually working on a quite difficult problem of communicating with multiple generative AI models to contribute to an outcome. So we are building some of our here using foundation models and we are building some of our AI models to be very customized for manufacturing for CRM purposes like customer360 marketing and sales. But at the same time we are Trying to solve the problem of how can we have specialized generative AI models which can communicate with each other each other's input to generate a final output. So we have heard about large language models. We are thinking about micro models now. Those micro models can now be much faster, cheaper to train, and can then look at the problem form from various specialized angles. Again, photography, you need to look about the light, the subject, the depth. So there are various parameters. Right. If I put all these parameters in one model, great. It will do the job. Or if I have small models for each of these, they will generate better results because they are working on a very small compute of very small parameters. And the probability distribution on those parameters are going to be much more effective in our hypothesis. So this is an experimentation we are doing is if we build a model over generative AI, it's not a model essentially, but it's a collaboration of multiple small models of generative AI to contribute to an outcome that will generate much better outcomes than one large model doing everything. That's our hypothesis and we are working on that.

That's absolutely fascinating. I guess it's like, do you want a gorilla or do you want a swarm of bees or something? It's like, I've got lots of little models working together. Good luck with that research. It sounds absolutely fascinating stuff. So, yeah, have fun with that and let us know how that progresses. All right, just to finish then, do you have any final advice for organizations wanting to adopt artificial intelligence?

Great. So that's a phenomenal question. The first advice is let's create a business case. I know a lot of you guys do. But really hone on to the outcome. I'm a big proponent of this culture in AWS called as working backwards. And working backwards essentially mean you release a PR article, like a public release article, imagining your product is already out. It can be six months from now, two years from now, don't care about it. And imagining the product is out, it's there and you are writing a PR article explaining how the product works. And then everybody in your team aligns to that PR article that this is what we want and everybody contributes to it and says this is what we want. And then have FAQs, as if you have FAQs on your website answering questions of all of your team members. Once you have this document, make that your bible and then work backwards from that. That what you need to do today to achieve that. That has created this plethora of innovation in AWS that actually triggered this idea of innovation. Because now anybody can send a worker in these warehouses, can stand and write a PR FAQ if there is an idea and then collaborate with the seniors to explain the PR to them what his or her vision is. His, her or their vision is and then go behind and plan what I need to do today to achieve that from one year from now, two years from now. So this is very powerful idea. Even if you don't like aws, you are anti aws. I generally like to take these nuggets of information, nuggets of patterns and practices from various organization and try to implement them because they are very powerful. The second thing I have really understood is from Jim Collins book is the flywheel effect. So flywheel effect is how the systems work together. Like for an instance, a good flywheel for AI is you start with databases, you take data from databases, then you use either data warehouse or data lake and build an experimentation environment which then leads to an AI product which then leads to again new data being generated which again comes in your database. So, so it's a flywheel effect, right? So it's a flywheel. So you have to constantly see the. It's kind of a lineage, but it's a lineage from a thought process of what is affecting what and how it is leading to. So understand these flywheels in your organization and understand how you can innovate things better using this flywheel. Because nothing in the organization is very linear, it is very affecting various other things and there are cycles of things. So understanding that before you start building the AI journey is quite essential.

I actually have to say I really like that idea of writing the PR article. Like before you start developing anything, just make sure the end product is really something sensible. That's absolutely, yeah, fantastic product development idea and I think especially applicable to AI. All right, yeah. So thank you for your time, Paul. Thanks for coming on the show.

Thank you, Richie. Thank you very much. It was great talking to you. Sa.