Vertical AI - IBM's Tactics for Enabling Builders from SaaStr AI Day 2025

Is vertical AI. There's a lot of talk in the market right now about the latest and greatest generative AI models that are out there. And what this particular direction means to IBM is taking the larger, more horizontal AI foundation models, like something you might interact with with a Gemini or a chatgpt, and making them more vertical specific. And that can be done in a myriad of different ways. But the goal here really is how do you take these foundation models and have them be applied to deliver industry specific expertise and deliver operational efficiency and be compliant depending on requirements of the particular industry or sector. And it really for software as a service companies and anybody who's building services or products with software unlocks a lot of unique opportunities to do and deliver value to your end clients in this fast changing space. It will help you deliver improved accuracy, deliver improved scalability and flexibility, and ultimately really be driving that enhanced customization that only you can provide to your end clients. And so why is IBM investing in vertical AI and the toolkits required for it? It really comes down to one major thing. We believe that the future of AI is vertical because today nearly all public data is represented in foundation models and enterprise data represents less than 1% of that. And so to us, that means that there's a massive opportunity for folks like yourselves to take foundation models, take your data, take your system integrations and partner with IBM and partner with other folks to be delivering that specialized, deep and targeted type functionality. And there's a myriad of different ways to think about this. If you need broad, general and adaptable question on what does a chief Product officer care about? Or what is the latest trend in, you know, XYZ science from the news, you've got places you can go to get that type of information. But if you want to specifically say I want to do and be leveraging work in drug discovery, that's going to require a deep expertise, a deep unique domain and data set and just a different approach to building these new AI features. And IBM is structuring our major platform, Watson X towards this reality, giving again software as a service companies and ISVs the ability to build on top of it in a way where you can take your data, you can take your expertise, you can take your software, take your platform and be getting even deeper. And so I want to take a quick pause on the value creation curve and sort of where we're at in the industry and why this is becoming even more important this year. There's a lot of talk in the past couple years about those first AI use cases or how do you leverage a rag pattern or do some fine tuning? How do you take a foundation model and have IT support your customers, service reps in their ability to understand and find questions quickly? Or how do you help a lawyer to do investigations on large amounts of data sets? These are things that are these amazing first use cases where people are getting a lot of significant amount of value from moving forward. We're starting to see a lot of people move towards things like IT automation, code development, digital labor and AI assistance and ultimately more agentic workflows which are conversations. We are starting to see a lot more in the comment, in the public vernacular related to AI. And this is really where that domain expertise, that specifics when we're talking about the pharmaceutical industry or even simply sales technology where you're going to have to go a lot deeper and again connect data with foundation models. And so IBM today is doing a lot of work in this space already. We're working with some of our strategic partners like ServiceNow, Adobe and Salesforce to take our WatsonX platform, deliver things like AI and autonomous agents for sales and service processes with Salesforce. But then we're also supporting folks like ServiceNow to leverage our models and governance, our platform for AI operations and trust and explainability to make sure that when ServiceNow gives you an answer powered by AI, it's consistent. We understand why it gave that answer and we believe that it's responsible and safe, if you will. But we're also working with folks that are more digital natives like Applause as an example where they're leveraging WatsonX to automate and optimize manual tasks related to software testing. So test case, rewrite, test, summarization and much, much more. And that requires again in all of these the ability to say, you know, we want to be providing not just a foundation model that can be providing, you know, general skill sets, but one that understands sales very well, one that understands legal really well, one that understands finance really, really well. So what is Watson X? What are, what is IBM bringing to the table? We've got several different pieces and parts here that all come together to deliver a full stack AI platform. So at the top, and arguably the most important is data. You're going to require connectivity to that information in order to have foundation models make use of it. And so with that, IBM has a lakehouse product called Watson X Data which can be deployed anywhere. It's built on Apache Iceberg and it's pushing the front end and cutting edge technology as it relates to lake houses in open source. We also offer models and we have over 28 different models available today from folks like IBM, Meta, Mistral and more. And these are really focused on being open source. They're also focused on being smaller language models for cost, affordability and ease of customization. We're also breaking out into spaces like Time series geomodels as well as code models which again play up into this future agentic reality where you're going to have to string a bunch of these models together to produce a particular outcome. And the focus of today's talk will be around InstructLab. And so we also have a governance platform to make sure that it's trusted and explainable as you're producing these outcomes. And AI models are taking on more and more responsibility from your employees today. But then we also have assistants and agents, again designed to pull these different pieces and parts together to be producing outcomes that ultimately drive value for your end clients and your business itself. So let me take a quick stop here with the WatsonX AI overview and platform. This is the one stop shop for an enterprise grade AI developer studio. It's where you'll find foundation models again by IBM, Mistral and Meta, but also the ability to bring your own model from hugging face. We even have the ability to run Deepseek, which I know is getting a lot of attention in the market today. But we also offer frameworks that are open source for building agentic workflows like Lane Chain, Llama Index and crewai and even frameworks for choosing the right rag pattern like our Flows engine toolkit. So this is the place where as you're building and you know technologies like agentic workflows into your software applications you would be going and interacting with first and it can be deployed anywhere. So you can put this on premise, you can put this in a cloud of your choice. We also have fully managed offerings sitting in IBM cloud as well. So when it comes to models, as I mentioned before, there's a lot of data that's in the public space that's already in these models, but not a lot of enterprise data. So choosing the right model that you can run affordably at 42x, lower inferencing costs and that 2 to 10 to 20 billion parameters range is really really we believe important as we manage to scale here in the new year. But also finding targeted use case this is for fine tuning is also very important because we're going to need you to take or you'll need to take a foundation model and add in information about sales or add information about you know particular insurance claim codes, etc. In order to effectively work within those workflows. And so as you're choosing those models, you're going to need to think about a few things. You're going to need to think about the thought process, is it open, is it performance, how customizable is it, et cetera, because this is going to be the foundation of that value and customization and really basis of your products going forward in many cases. And so what does it mean to customize those models? There's a few different ways to kind of think about it and I'll walk through some of them right now. But your base model should be transparent so you know what content is inside of it in the first place. And then you're going to be adding data into that, so you're going to need to know what you're mixing it with. But then ultimately the result is something that your customers or your enterprise will consume. And there's a few ways which I'll walk through here to get the data to be interacting with the foundation model. The first, which is a really common thing that I'm sure we've, you've heard quite a bit about today, is the rag pattern. And it's the ability to essentially refresh your foundation model with more relevant data or more up to date data. So that could be embeddings and storage of HR policy, or could be specific information about a particular industry like a pharmaceutical company or the oil and gas industry as an example. But there's also the ability to update the model itself so that it understands your data, but also potentially skills as an example. Now fine tuning is a great way to do this today, but one of the challenges with it is every single time you go fine tune a model, it ends up becoming a completely new model. And you have this proliferation of tens of twenties of different models that you always have to be updating and changing. You'll see if you go on to some of the different platforms out there, like hugging face. There are thousands of different llama models as an example that have all been fine tuned for various use cases. And so IBM thought a lot about this, specifically IBM Research, and we came out with a way to train your model called Instruct Lab. And what it does is takes the LLM and in a secondary motion adds skills as well as knowledge to the model itself in an incremental way. So you can keep one model and without the proliferation and be consistently adding additional knowledge and skills, sort of like humans do, without the need to sort of, you Know, clone it and repeat it, so on and so forth. So let me talk a little bit about how this actually works. So with fine tuning you need quite a bit of data and you with InstructLab need quite a bit less. Because what you're going to do is give a few examples. And I believe the metric is one to 1,000. So you give one data and it'll go and create a bunch of synthetic examples and that will be used with a larger model or the teacher model to create all of that data sets with a critic model that ultimately teaches a smaller model. And what this does is you'll take a much larger model that's 70 or 400 billion parameters, and have it educate and update the knowledge and skills of a much smaller model somewhere in the 2 to 10 to 15 billion parameter range, which is much more affordable to run and also much faster in many cases as well. And so as an example, what we've actually gone out and done is done this instruct tuning on IBM's Granite series and with a 7 billion parameter model, we have a few examples here of how that lightweight model was able to not only effectively be more performant, but also meaningfully cheaper than much larger models. And so you'll see here as an example in column 2, Granite 7B versus the llama 370 billion was able to be 66% cheaper while almost 10% more performant. And there's multiple examples of this where we were able to kind of come together with folks like yourselves, add that unique data and so, you know, move forward. And so at a very high level, what we're seeing is IBM Granite and Instruct Lab are able to deliver a 98 and a half percent cost savings and in comparison to traditional tuning methods, a 35 week time savings. And so that means that customization of these models and the tuning and training of these models can be done meaningfully faster and meaningfully less expensive so that you can use them in many, many more places. Imagine being able to save 35 weeks instead of tuning the model. During that time you can be figuring out different ways to be leveraging the model that you create, that is delivering again that particular outcome, be that climate science or retail supply chain optimization as an example. So today we're building this into our Watson XState AI platform and we're making it easy to be doing this type of work. From advanced data ingestion to data model evaluation and audit readiness. We're able to put all of this type of functionality in a low code, no code way, where you can even be interacting just via a GUI to be updating models on an ongoing basis, leveraging InstructLab. Now you can also do fine tuning without our platform and you can also leverage rag. And frankly, we recommend that you evaluate and leverage all different options depending on your use case. Because in some cases you might need a RAG pattern, in some cases you might need to fine tune, but in many cases, especially with the new agentic realities we're walking into, you'll probably require all three. And picking and choosing those is something that we want to be providing as a part of our platform. So you have that flexibility to pick the right motion that works for you. And so I recommend that you go out and start to actually interact with this. There's two main ways you can do so. One, there's a QR code here that will be up next that you can take a screenshot of and dive into for our WatsonX AI demos. And that's actually within our WatsonX platform. So you can start interacting there, but you can actually Even do an InstructLab project on your laptop. And this was open sourced. And so you can go out to the InstructLab GitHub and be downloading this and interacting with this toolkit there as well. So here's how you get in touch with us to dive into this. Somebody from IBM will reach out when you fill out the preview interest form and we'll talk you through what we've built, how it can apply to your use case, and how you can ultimately be taking this. Amongst many other technologies IBM provides to be providing additional value on top and driving towards vertical AI. And so if I can leave you with three things, the vertical AI unlock is really the basis of how IBM and our software as a service partners are going to market. It's the place where differentiation occurs and ultimately where you can go make money. Customizing foundation models is a challenging thing to do and there's a lot of different ways to be adding that vertical or domain expertise into the foundation models. There's a lot of ways to do so and IBM has unique ways like Instruct Lab as well as common ways like RAG and fine tuning. And ultimately the goal is accelerating differentiation. So if you have a set of data, a particular use case, we highly recommend that you reach out and would love to sit down and talk about how we can be applying our platform to be accelerating your outcomes. So I believe there's a little bit of Q and A time here.

Yeah, let me grab some questions from the live stream. All right, first question we have from the live stream and then if you guys are in the zoom and have any other additional questions for Nick, just drop them into the chat. But we've got a few from the live stream as well. First question being can you share some examples of successful applications built with IBM? It's a two part question, so let's maybe do that one first.

Yeah, absolutely. So let me.

And maybe like SaaS applications since most folks here.

Yeah, absolutely. Let me go back to this slide I touched on briefly. So yeah, you know, here are some of the common top use cases and there's a bunch more. We haven't quite gotten to a place where we've signed the dotted line about marketing just yet. But Applause is probably one of my more favorite ones because these guys work in the software testing space. Super domain oriented, lots of opportunity to provide automation, leveraging the modern foundation models. Right. And so we're working with them to take our platform and to in particular technologies, you know, be able to provide automated testing. And so they're using our platform to say hey, this is the foundation model, these are the languages we want to be producing an outcome on. And here's how specifically we want to be setting up a test case and here's what good looks like and they're bringing that again that domain expertise. We're offering the tools and kind of coming together. And one of the fun parts which wasn't really mentioned today is a part of our build motion. When we work with software as a service companies like Applause, we also have experts that come in and we'll sit down with them and sort of do those do that building together with you all. So we've got folks who are investing in and spend all day helping others leverage our technology to build new products and features. Any questions about the rest of these up on the screen?

There was a follow up question somewhat related so I think you can stay on this slide is for folks like this, how do you IBM help help these SaaS companies stay competitive since the rate of change is just so fast, right in AI?

Yeah, yeah, it's a good question. So one of the really exciting parts about working for IBM, I have a fairly consistent cadence myself with our IBM research group. And so IBM spends almost $7 billion a year like with a B on investments in research and that goes to quantum, that goes to artificial intelligence and so much more generative AI in particular has been a big push for us. We've got an MIT lab, we've got a whole bunch of stuff we're doing in that space. And so working with your account team here at IBM and asking those like, ongoing questions, I recommend as a best practice for my folks to have monthly cadence calls with their partners to say, hey, here is the latest and greatest. Here are the things you should be considering. How, how are you going on your roadmap? And we do our best to be applying a white glove treatment there. Also, we have a bunch of great content out there from IBM Research. If you follow our LinkedIn pages or our blog, I believe it's IBM think has a whole collection of there. So we have maybe too many channels to go follow.

For sure. Okay, and then last question on the live stream here is for folks maybe just starting a bit earlier in their AI journey. Right. Like, you've gone through a lot of the content here, but I guess just simplify it for us. What's like the first steps they should be taking to really catch up a bit, you know?

Yeah, yeah, yeah, Good question. I would say there's like two schools of thought. One is you just gotta start building. I cannot recommend that enough. We're seeing this whether you're looking at like, Visa has posted a bunch of really interesting articles about this. You just gotta hit the ground running. The technology is going to continue moving, but you got to get out there and just start using it. So doesn't matter whether or not it's a use case you even deploy. Especially in the last two years, we saw a lot of people just go build for the sake of building. We recommend that on the second piece there's a unique sort of. It's more philosophical. We're at a moment where value and the meaning of value is changing. You can't just come in and provide integrations to different technologies and easy to use UI anymore. People are expecting more because they can go to ChatGPT or Gemini to get these immediate answers and to do so in a pretty interactive way. And so I encourage sort of a design thinking process or a first principles thinking process where you go back to what your core value is and then start to build technologies into what that is. There's data out there that you own, there's system integrations you own, relationships that you own, patents you might own that are your core value. AI is supposed to be an accelerant on those. And it's getting tougher out there, but we also think it's going to be a lot more valuable for everybody.

Cool. All right, Nick, I think we are approaching time. So for folks who have more questions, what's the best way to get in touch I know you shared earlier, but what's the best way to get in touch with you? Slash IBM on this?

Yeah. I highly recommend reaching out on LinkedIn. The QR code, which I'll flash back up here, is a good way to talk a bit about InstructLab, but it'll also get you into our process where, you know, we'll get you talking to the right folks. So. Yeah, either direction.

Yeah. Awesome. Yeah. Thanks so much, Nick. We'll see you at Samster Annual with the rest of the IBM team. We're looking forward to it. And thanks, everyone, for joining this session on AI Day. We've got two more sessions to go, so we will see you guys there. Thanks again, Nick, and see you guys at the next sessions.

All right, thank you all so much.