How to Incorporate AI in Planning

All right, welcome to whoever is out there with us right now. We're going to get started here in about three minutes or so. But if you're listening right now, if you've joined us, let us know where you're coming from, let us know where you're located. We're really excited to have you. We'll be quiet here for a minute or two before we get formally started. We got a couple minutes before this starts, but, but just figured we'd jump on so everybody knows we're here. And as I mentioned before, let us know where you're coming from in the world. And if you want to let us know one thing you're most excited about for today, go ahead and do that. All right. For whoever's out there, we're going to get started here in just a minute or two. You know, some of you may be on YouTube, some may be out there in LinkedIn. In a minute here, I'll kick off our 30 second countdown video. If anyone's seen some of my webinars, they've probably been tortured by it before. But you'll get tortured by it one more time before I introduce Justin here and we jump into some really exciting questions in the world of AI and planning. All right, again, if you're out there, if you've joined us a little early, let us know where you're coming from. I'll ask that question again here once we actually get started. But I'm going to go ahead and kick us off here with our Countdown.

Foreign.

Well, welcome for those of you that are out there on LinkedIn or YouTube or wherever you may be joining us from. If you're, if you're out there, please let us know in the comments where you're coming from. We'd love this to be interactive, so feel free to put a comment out there and let us know where you where you're at at the moment. I want to start here by introducing my guest. I have here with me Justin Croft, who is the Solution VP of Solution Architect at qbit and I'll give him the opportunity to spend a minute and introduce himself to us.

Yeah, Paul, thanks for having me. I'm excited to be here. As you said, I'm the leader of Solution Architecture at qubit. That means I spend a lot of time doing designing and requirements gathering of solutions that we implement for our clients in the planning and AI space. My background is fpa. I spent more than a decade in corporate financial planning and analysis doing a variety of different funding, predictive models, planning, you name it, we did it. And really kind of that drove me towards data science because I felt like there was, there was more to see, more to analyze, more to find. And now I've got the real joy of being able to put those, put those two things together. FPA and finance and artificial intelligence. So glad to be here.

Great. We're really excited to have you. Appreciate you sharing that. And I do know we have some people out there on LinkedIn. Just know we're dealing with a little bit of a delay, so hopefully your comments will come here in a moment. But don't be afraid to ask questions. Just know we have a little bit of a delay in the feed that happens from time to time on LinkedIn. So I know I looked at my phone and I could see some people out there and see that they've commented, but it hasn't come through to stream yard. So just be aware though, we'll come through. Hopefully that lag will shorten up a little bit. But it may take us a minute to get back to you because it doesn't always. I'm. Fortunately, not everything's real time like we think, Right? Why hasn't AI solved this? Justin, he's like, do everything yet? Dang it. All right, well, let's jump in. The first question I'd like to ask you when it comes to AI and everything we're seeing, how are you seeing companies use AI in planning today? Let's start with that. What's the kind of the main use cases you're seeing?

The main focus and has been for a number of years, is really the idea of what I call traditional time series forecasting, analyzing a historical set of time series points. And that might be sales, but you.

Just don't feel like you understand the mechanics. How do you think about that with the rise of AI? Right. Because if you look at generative AI, very few people really understand what's going on in the background. It's just returning an answer. And often that's even the case with some of these models. So how do you balance that and get comfortable with the fact that this technology plays a bigger and bigger role? There's going to be more real concerns and situations where it's going to feel like a black box to many people. Doesn't mean it can't be explained, but maybe we just don't have the experience or we don't understand it well enough. So how do you balance that? Yeah, it's all becoming, I think, I.

Think a lot of that has to do with the business process. That you wrap around the predictive capabilities. We really approach it as statistical modeling. AI provides a foundational forecast. Right. So this is your baseline, and then on top of that, you've got a human who comes behind and layers in adjustments, overrides. There are always going to be things that the model doesn't know about or doesn't understand, particularly when you're looking at things like new products, product line extensions, geographic expansion, pricing and promotion changes. Those are all examples where you really need a human to augment whatever the model is putting out just because the model doesn't have enough context. And so we, like I said, we think about it as layers of a cake. We where the vast majority of the work is being done by the baseline statistical forecast, and then the other 10%, let's say, of the work is being done by the analysts who come behind and layer in those adjustments. So within that controlled business process, I think it's easier to say, well, this number makes sense, this number doesn't make sense. I'm going to adjust this number. Maybe the model doesn't appreciate all these other factors we had.

Makes a lot of sense. I had kind of, as I think about that, and here you say the adjusting and thinking about things. I had a behavioral economist on one of my podcasts, and she talked a lot about how it's technology and people. You got to be able to have that validation, at least someone look at it. Because there's always subjectivity because we're predicting forward. Yes, historical is accurate. Hopefully.

Hopefully.

But you know what I mean, it's historical. It's happened, it's taken place. When you're predicting forward, nobody's certain what's going to happen. And so there's always going to be that judgment. I think that's, that will always be the case. No matter how good AI gets is there's still that element of judgment in this whole process.

And when, when we have clients who get concerned about that, that's when I, I really pivot towards the idea of scenarios to say this is what we think is going to happen under this set of assumptions, we can make changes to those assumptions and you're going to get maybe a slightly different forecast or dramatically different forecast. But this is one potential version of the future among many. So let's talk about what really drives your business and what levers we want to pull within the forecasting process to come up with a final set of numbers.

Yeah, I, I do like that. And so, you know, we've been talking about the planning, some different models you know how to think of generative AI and you know where you're seeing some use cases. I'd be curious is we have some people out there and before I finish asking this question, I just want to let everybody know I'm keeping an eye on LinkedIn to see if you have any comments. For whatever reason, Stream Yard's comments are not showing up on our end today, which is the platform we use to integrate with LinkedIn. So I just want everybody to know that I know we've had a few people join from in India and there's people out there and so we're just managing that as best we can. It's the joy of technology. So we're not ignoring you if you put comments. I'm trying to keep an eye on it, but it's kind of hard to manage three different things. But I'll do my best. Here. So with that said, I'll get back to my question, but just want everybody to know who's out there and listening. We're dealing with that and a little bit of a lag. But how, how should people think about AI if they want to get started with their forecasting? Right. Sometimes it could just be overwhelming. You're like, all right, I know this technology could help me, but I just like my Excel, it works and I'm happy, but I know I probably should be doing something. So how do you, how do you get started? How do you think about that? And what do you tell your clients?

Yeah, yeah. To build on something I mentioned earlier, right. When I was in fpa, I was looking at rows and columns in a spreadsheet, trying to come up with a forecast. And I always felt like there was another dimension I was missing. There was another set of relationships and patterns within the data that I was, I was really missing. And that, that drove me to leave FPA and explore the whole data science side of the business. And now I'm able to kind of combine those, those, those two interests productively. But I think there's, there's this idea of really wanting to analyze your data and find patterns in a way and at a scale that a traditional spreadsheet based approach can't do. So, so that's the first thing is you've really got to have a business problem that requires that level of rigor and cost and expense overhead of bringing predictive to the problem. Right. Lots of companies are fine and Excel, our clients typically are dealing with what we talk about as multi dimensional problems. They've got multiple departments, multiple locations, thousands of products or tens or hundreds of thousands of products, products that they want to analyze and forecast. And you can't do that in a spreadsheet. The most I've seen a company do is about 10,000 products in a spreadsheet and they were pulling their hair.

No, thanks.

Yeah, so taking that multi dimensional problem out of a spreadsheet, right, that's, that's another area that you really need to look for. That's a signal where you need to migrate towards another tool. And generative AI can help you get started, right? If you go to any of the generative AI platforms and ask it to write Python code for you to come up with a forecast, it's going to do a great job. And then you can take that Python code and just use that as your first entryway into a statistical forecast and AI forecasting model. So the barriers, from a technical standpoint, odd as it sounds, with all of the hype out there, the, the technical barriers are really lower than they ever have been before. It's easier to get started with this than it ever has been. You just need to look at your business problem, look at your data, make sure it's right to apply AI. Does it make sense? Is it worth it? Right? Is the juice worth the squeeze? People say, and then find the technology that makes the most sense for you and start to apply it within a planning process.

I liked a couple things you said there, right? Is the juice worth the squeeze? That's something everybody has to think about. If you have very little data, really messy data, is that the best use of your time to try to figure out how to use it? I also really appreciate that generative AI, although you can't just feed it your data yet and always expect, you know, expect it to output something. One, most of them are public, so don't feed it your data. Two, as you mentioned, a lot of approaches we could take is, hey, I want to try to statistically model this. Here's what I know about my data. You know, could you write a script in Python that would help me get started? And you could continue to ask it. You could ask it, okay, what's it doing to give you explanations? I think the thing to remember as you're getting started is generative AI isn't something where you just ask it a question and move on. You got to continue to prompt and, and push it. And you also have to, you know, kind of be able to monitor it and make sure it all makes sense. But I really like how you said there's ways the barriers to Entry are lower. Like, you don't have to know Python at the same level to do some basic things today as you used to.

You don't, you don't. Now that said, the mistake that I see people making is to start with the technology. You really need to start with the business problem and focus on what it is that you're trying to solve. You know, within, within planning, it's a little easier because it's pretty clear you're trying to take historical data, historical numbers, and come up with a forecast along the same set of dimensions. But do you need to incorporate external predictor? Do you need to adjust for outliers? Do you need to cleanse your historical data? Bad historical data is something that we deal with every day in the time period post Covid. Right. There are a lot of companies that we work with used to have five, six, seven years worth of data, and now they have two. So how do they adjust for that?

Well, you know, and then you talk about, even if you have good data, do you really want to use a Covid year as your baseline?

Right.

Right there. You either got to make a lot of adjustments or assumptions or. Because past is not necessarily a predictor of future. Sure. Can it help you? But there are times when it's probably a poor predictor of the future. Not times that it's probably. They're times that it is. Let's be honest.

It is. Yeah. Just. Just to build on something you said, do you have enough historical data?

We've.

Excuse me, We've got customers with years and years worth of historical data. We've got customers with weeks of historical data. So you really, you really want to be in a sweet spot there where you've got enough historical data to make it worth your effort.

Sure. And that makes a lot of sense.

So.

You know, one thing I want to ask, when you and I chatted, before we, you know, jumped on this conversation, you had mentioned something that I thought was interesting and really stuck with me. You had mentioned that when you're incorporating AI in your planning process within your finance department, within fpa, don't think of it as just a project. Think of it as a project program. Can you elaborate on that? What did you mean by that? And what's the danger of thinking of it as a project?

Yeah, yeah, absolutely. So I really like this concept. And, and a, A project to me has a beginning, a middle, and an end. Maybe you're, excuse me, maybe you're implementing a system or doing some sort of integration and it's got a beginning A middle and an end. At some point you're done. Right. And everything just works and you can go on about your, your life.

Yep. Building a house.

Yeah, absolutely. A program requires ongoing care and feeding to be effective and to be trusted. And, and I really say all predictive should be approached as appropriate, particularly within the Office of Finance, because you're, you're never done. There's always more that you could do and there's always more that you have to do in terms of understanding and explaining the model and creating the next forecast and the next foretext. So models have to be cared for and fed. They have to be reevaluated periodically, particularly if you're incorporating external data that has to be evaluated and reevaluated for its relevancy and additional input and accuracy in, in your models and in your process. So you really need to think of this as a program and it's, it's part of what you do on an ongoing basis. Right. It becomes part of your, your planning process.

I like how you said that and that that makes sense to me. So I'm curious. Obviously you went from FPA to the data science side to the consultant where you're helping companies get better at FP&A, incorporate the data and move beyond, you know, just Excel, as I like to say. Not that you can't do statistical modeling in Excel, but you know, there are, we have planning tools for a reason and we continue to grow and scale.

Right.

And so I would love to know, you know, maybe one or two real success stories you've seen of companies using AI in their planning. Could you share some examples where you've seen it make a big difference for them?

Yeah, yeah, absolutely. So several come to mind. One was a household name bakery, baked goods company that came to us needing some help, and this was during the pandemic, needing some help forecasting out their future sales and really handing out their business. And they came to us with some preconceived notions about the data that really drove their business. They thought that certain, certain data elements, things like the number of miles driven, the price of gas, how mobile people were, that that really drove their business. It was related to the sales of them products. And we said, great, let's, let's look at that and do a deeper dive. And what we found was these, these historical data points, these levers that they really managed their business by, were not at all related to sales of the products. And so we had to kind of kill a sacred cow, so to speak, in some sense there because people were very protective of this idea. And so we had to introduce some new predictors, some new external data that did have relationships to sales and to products and, you know, replaced what they had traditionally used as drivers of their business. When we did that, we not only got a forecast, we got a much more accurate forecast. One that they could really depend on and one that they could understand and drive through business. What we found was distribution channels and in particular Walmart and large box retailers that really drove their business. And it wasn't sales and convenience stores and miles driven in the price of gas. Consumer behavior had shifted and part of that was the pandemic, but part of it was simply time and the evolution of culture. So I think that was a real success because, because we were able to, to not just say, hey, this doesn't work, but hey, let's look at this slice of data and let's manage your business this way. So that was, that was a lot of fun to do. A lot of fun to work on, I bet.

To be able to help them completely change the way they think about things, what inputs go into it, to have a better output. That's what we all want. So I appreciate you sharing that one. You know, there's a question that I wanted to ask you as we've been talking that I've been thinking about in your journey. You know, we have a lot of people now and I get asked this question from time to time, hey, do I need to get a data science degree in FP and A? And with all the changes that are rapidly going on, what would you advise people? How do you think of that balance of how you know, how much you need to know the data and the data science and modeling, as we move forward and we see more AI, more statistical, more predictive type of stuff, what do you tell people or how do we think about that in your view?

Well, my answer is somewhat biased because it's kind of reflective of where I came from.

Sure, but, but we're all biased. That's okay, we'll take it. Nonetheless.

What'S not going to go out of style is really understanding your business. Technologies change. Gen AI may sweep away statistical modeling, who knows, but, but what's really never going to go out of style is understanding your business and the way that your organization works and how you make money and, and if you understand that you can apply whatever tool or process is, is, is relevant at the time and be successful. So I'd say focus on the business first. Focus on the business fundamentals. Understand how your company Competes and how you make money and how you lose money. Know what those external predictors are? What are those levers that really drive your business out there in the marketplace? And then you can apply technology to a planning process.

I, I absolutely love that answer because the thing I tell people so often, whether it's applying technology, whether it's applying finance, start by learning your business. Understand the operations, understand the flow of a dollar form, how does it flow through the entire process, both for expenses and for different costs. If you understand the flow, you will be so much more helpful in meetings and you understand the operations and a little bit of strategy than just being able to go. Our margin's 36% and we'd like it to be better. Okay, well that's nice. Well, how do we do that? Well, we know that our biggest cost is X and we know there's been a lot of competition in the marketplace. Can we get a better, you know, better price from a dealer and being able to actually have that? We know that so and so has come into this space and the more, you know, where you can contribute to those conversations and help drive the business forward because. Right. Planning at the end of the day is all about trying to help a company achieve its strategy.

Absolutely. Yeah, absolutely. And you know, things like understanding maybe even though the way that your, your products mix changes over time.

Right.

If you just focus at the top level, revenue may look great, but maybe your products are being sold in low margin channels at a higher rate than they used to be. So understanding those sorts of dynamics and that, that multi dimensional aspect that I spoke about earlier is, is really important as you transition to more advanced types of model.

Well, great. I'm gonna, we're gonna wrap up here in a minute. I'd normally do a few minutes of Q and A, but for whatever reason, comments are not coming through on our end today. You're welcome to try to throw in a comment, but we haven't got one yet. And I know a few people have put them out there. We had a few people tell us they were from India and things. So we'll, we'll hang on here for a minute or two, see if any comments come through. We'll address those, but otherwise we're going to go ahead and wrap here. So I'll give you a last word. Justin, any last thoughts you'd like to share with the audience?

Absolutely, I would, I would really encourage everyone to explore how to apply AI within a planning process. As I said earlier, the technology, the barriers to entry are lower than they ever have been before. So this is something that you could explore and implement in fairly quickly with fairly low risk. It's, it's achievable, it's attainable and it can really add value and, and add efficiency to your processes. So get started. There's no reason not to be doing this.

I think that's great advice is get started, figure it out, improve your process. I love. There's a graph I love to share that shows companies that are using the cloud and, and AI to ML in their forecasting. The business is much happier with their.

Forecast.

And that that's usually a sign of other things as well that they're really thinking, how can I use technology? It's not just that, but in general rule it's a lot better. So I think those are great parting words and we'll just hang on here for about another minute or so and then we'll go ahead and close the session. But thank you everybody who's joined us. Justin, thank you so much for taking 30 minutes and chatting with the audience. I know I learned a few things and really enjoyed talking about about it because I'm excited for the future. It's going to get better.

Well, thank you for having me, Paul. This was a lot of fun.

All right, great. Well, we're gonna go ahead and end the stream there, but if anyone has a dying. A question that they're dying to ask and they didn't get it, you could always email me. I'll send it over to Justin. We'll get you back an answer on that. So thanks.