Abhas Ricky, Chief Strategy Officer at Cloudera, identifies four trends reshaping enterprise data strategy: the $100 billion compute scale of major cloud providers, enterprises treating data as a strategic asset for AI training and retrieval, the link between data quality and AI output fidelity, and data sprawl across teams creating security and cost problems.
What are the most important trends you're seeing in the world of data?
Yeah, so there are a couple of things that we're starting to see. So as you know, for the last 10 odd years, majority of the focus has been about cloud applications. There wasn't technology in the last 40 years where you have three very large providers, each of them with $100 billion of compute or more growing at 30, 40% CAGR and that kind of fuel the data application business to next levels. But that also brought in new challenges around data security, data governance, data lineage and all of those capabilities. So over the last five years or so, when realistically the Transformer Library paper came out from OpenAI, there has been an increased focus on data as an asset. And therefore majority of the large enterprises we speak to, or I speak to on a weekly basis, they all want to say, hey, I want to be able to trust the data that I'm going to use and to train my large language models on. And whether it's agency applications, whether it's rag applications, whether it's fine tuning, the core premise is you want to be able to trust the data that you're going to use for any of your AI applications. And I do think that the trust in the data is a big, big subject that's coming through. The second thing we're starting to get through is the fidelity of the outputs that we're starting to talk about. Oftentimes that depends upon the quality of the data sets that you have. And therefore in AI parlance we call that as enterprise context. There's an increased amount of focus on getting the high value enterprise context, whether that's for AI applications, whether that's for easy applications or there are, and I believe the third one really is to the original point that I said, which is there's a lot of data sprawl that is happening because remember architecturally we went from data lakes to lakehouses to now a slightly different architecture for AI technologies. But along the way there have been application developers, there have been AI practitioners, there have been SQL developers, front end and back end, and they've all created their own systems and tools. And there's been a lot of data sprawl that has happened and that creates issues around data security, that creates issues around cost. Total cost of ownership is a big element. So that is something that is coming to the fore more and more. And large enterprises are trying to tackle that with the best tools and services they have at their disposal.
Those all seem like incredibly important, well, very important things. So yeah, you want to be able to trust that your data is going to give you the right answers and.
You need to be able to find.
It rather than just working out which application it's sitting in or which database it's sitting in. So you mentioned the idea of data as an asset there. Can you just tell me a bit more about what that means?
Yeah, so technologically a lot of the people have said they wanted to get to data mesh architectures and that's a source of capabilities, right? So that includes query federation, for example. That includes adding attributes on the data itself so that you can treat data as a product. Because at the end of the day, that's where majority of the application developers want to get to so that their job becomes easier. But on top of that, I'll give you a real example. There's a very large bank who I work with. They have close to 1 exabytes of data under management and that's a very large number structured, unstructured images, all kinds of data. But for two, three years back there was all storage and you could pull some of it out for analytics, you could pull some of it out for your next best action or, or your sentiment analysis or whichever the use case might be. And then you worried about making sure you were compliant with the policies. If you're a regulated industry, in this case they were. But also elements in Europe as you know as GDPR or in California, CCPA came through the fore. So a lot of the value was around making sure you can use the data and parks to get to a specific outcome. But with the onset of AI, what has happened is the data has been completely the primary focus because of democratization that is taking place. For example, if you were a SQL application developer and I was a data scientist two years, three years back, you would have to come to me every time you wanted to change a parameter on a model. Now you don't need to do that because you can take the enterprise context, AKA data asset and apply it to the large language models without even having to come to me as the AI practitioner or data scientist. So you don't need to evolve or upskill necessarily and you can get different outcomes for different use cases with the same data set. So the data is becoming much more powerful because now you can use that as anti practice context and apply large language models or any of the new AI frameworks that might be on top of that. So I do think that there's an increased focus on the quality of the data, as I said, but also making it available, getting into the hands of practice practitioners is equally important, and data as a product definitely helps you get that.
