AI Unleashed: A New Era of Budgeting and Strategy

Hello everyone and thank you for joining us for our thought leadership webinar AI Unleashed A New Era of Budgeting and Strategy. I'm happy to introduce our speakers for today, Rafael Baptista and Ted Jackson. Rafael is the Director of Strategy and Innovation at Buncombe County, North Carolina specializing in local government innovation and strategic planning. He holds a Master's from UNC Chapel Hill and and actively contributes to Local Government Leadership Forums. Ted is a co founder of clearpoint Strategy and an expert in strategic management and performance methodologies, significantly shaping the field through his work with the creators of the Balanced Scorecard and clearpoint software development. For the purpose of this webinar, our discussion on AI will focus solely on large language models such as ChatGPT, Bard, Grok, Claude and similar.

Here's what I'm trying to get across Please make this email more empathetic or a lot of times I use it to ramber check me. I didn't use ChatGPT on a recent email I sent out to our entire leadership team at a mid spell Pittsburgh.

And two weeks later my cool conversation about budget to thinking about what's changing and what's driving the budget and how that might be a concern to people and you know, thinking about the the pieces that would go with it. It's completely different message than you know, what might be inside of your budget book. Yep.

And just something to note here on the thing. I think it's the second to last paragraph it says in close with this email is a visual guide help illustrate these changes. There is no visual guide so this is just a good reminder. Sometimes ChatGPT well makes stuff up.

Yeah, it's a good. Yeah, absolutely. You should, you should always be proofreading your your work to make sure it's going to to capture what you need in it. I've. I've used this quite a bit to to write emails and then absolutely you go back through it and say well that recommendation is not that great of a recommendation so you know let me change that one. Or this person might not appreciate the way it's written so ask it to draft it a second time.

Third time Ted since the last time we've talked I actually got the Jet DP Ford plus account, the paid account and I learned a tool in there that I didn't know which is you can actually go in there and give information about yourself and your style. So I actually went into ChatGPT and said hey this is generally how I write. This is my style. I like to have bad dad jokes. I like to be A little less serious. Making points across. What I've discovered is it actually now writes bad dad jokes. And some of my emails for me, that is.

That is pretty good. Pretty handy.

Yeah. Because one of my challenges was I used chat GPT and it just, it wasn't in my voice and it was obvious that I didn't write that email. Chat GPT wrote it. So now it does a better job of writing in my voice, which is nice.

Well, at a holiday party I. I was talking to an oxygenarian.

A what?

An 80 year. And he was actually 88 years old. We were talking about chat GPT and AI in general. And he said he was asked to write a chapter in a book. And I said, you know, chance you'd be taking to help you with that. That. And, and so, you know, into the party the next morning I got up and I tried to play around with it to give him an example because he had never been in it. And I said, write it in this. He's written another book. And so I said write it in the style of Richard Holm. And it immediately just spit out a chapter because I gave it the subject and everything. And then I said who do you think Richard Holm is? And it said I don't know who that is but I went ahead and made this up. And so then I referenced the book and said oh, based on the book, he is an international such and such. I will now rewrite it in this style. But, but you're right, you have to be careful and thoughtful about what you feed this thing and how literally you take it. So I would recommend reading those dad jokes over before you before you send them out.

So I'm going to venture to guess that most of the audience is like me and Dustin, works in communications but works in some form of data or strategy or performance. So should we move into how we can use ChatGPT to analyze some data?

Yeah, let's look at it. So this is also from Hendersonville. It is. It is data about their water web. So let's, let's see if I can cut and paste this data in quickly. Let's see. I don't think it really likes just typical cut and paste. Oh, there it is. So did do that. So let's see, see what it thinks. So I just cut and paste in the table if it went in as an image or no. Okay, good looks at it. Certainly got a little bit of humor to it today. So there you go. So looked at it and it could tell that some information our actuals were less than what we budgeted for revenue and so it's for I guess the seventh grade level. It's comparing us to high school grades or middle school grades I guess in that situation. So then we could even ask it from here, what should do. Also found that sometimes Chat GPT doesn't care as much about punctuation and even could pick up spelling so that it might not have a problem with your Pittsburgh spellings.

Bonks of captures the fact that I misspelled Pittsburgh and Pixa said I'm good.

So let's see, can it figure out what to do about these results? I guess it's still talking in seventh grade level.

Well, how about we reframe the prompt and ask it. Please analyze the results from the perspective of the utility system director and give three things that the director should do something like that. See how technical it can get.

Oh yeah, this is no longer at the seventh grade level anymore. So one of the nice things that I've really enjoyed about ChatGPT or any of these language models is that they persist like you can have a conversation and if you don't get the answer you want, you can continue to ask. Whereas a typical search tool like Google, for example, you have to re ask the question and start from scratch. And so in this case, you know, we put the data in way up in the scroll and we're continuing to ask for information about it. So I think for, you know, I think the data is still that like 80% of people haven't tried to use AI before. So using it, that's one of the things that you can, you can find is a great benefit in the process is that you could, you know, ask how do I develop a capital improvement reserve fund? And just kind of keep asking questions about it as if you're having a conversation with it, which is really nice.

And these are pretty good responses they gave. I mean it, I mean probably not a utility director level response, but pretty dang good.

I'll send you over these chats, you can pass them along to the utility director. I guess we'll have to send it over to Henderson.

Yeah. So you know, this shows kind of the way ChatGPT can analyze data. It's pretty impressive, right? So some pretty good stuff. It's really helpful if I'm not a subject matter expert to get up to speed a little bit. But my understanding is you've also played around, built some stuff into clearpoint that can do some of this analysis, right?

Yes, I am happy to show you that while you're talking, I Thought I would test out one more feature of chat. GBT is just have it tried to build a chart based on the, the data here. We'll give it a second to do that and then I will jump into clearpoint and show you. Basically in a similar way that Rafael talked about putting in some of his information to get emails to write in his writing style. This will be, We've, you know, plumbed the language model to pull information through clearpoint. So this chart, again, this one maybe is not perfect because budget and actual, I guess they put budget behind actual in a way. Maybe we've asked for a different chart, but let's jump over to clearpoint. Seems to have timed out while I was, while we were talking.

And while TED locks in, folks, don't forget you can throw questions in the chat and we'll try to answer them.

So in clearpoint, for those of you that don't know, and I imagine most of you do, it's a tool that tracks your strategy and helps you, you manage that, you know, either at the enterprise level or you could track an annual plan, your projects, et cetera. So you could track your, your objectives, your measures and your initiatives or projects. And if we look down at, you know, one of our measures, we typically see data in the measure and the data is built off of a data table. This data table is relatively simple. It just has actual and target data every single month. It has a variance calculation associated with it. That's a simple subtraction. And then you get to also turn it into a percent. And we can look at our data over time. And then, you know, what you're supposed to do is analyze this. And so this isn't all that sophisticated of analysis and recommendations for our fake airline. So what we've allowed you to do is we've put an assistant in the tool to, to analyze the data for you. And so what it knows how to do is look at the chart, look at the data table underneath it, and then be able to, to read it, right? So it can tell you some information and they can say, oh, here's some potential reasons for that drop. And, and so I could copy that analysis directly in, take away the, the final question there, save my results. And then what I could also do is get some better recommendations. And so it can, you know, investigate seasonal factors affecting the December revenue, or review operational issues and assess market conditions. And, you know, these are pretty good ideas. I could add them as action items, assign them off to members of my team to do it. But if I was responsible for this I could even ask, you know, how would I investigate seasonal factors? And so it just tells you how to do something. So sometimes even if you don't really understand what you should be doing, you can see it here. And we could either turn these into action items or just copy the full conversation down here and we can delete, you know, pieces of it if we wanted to, let's say from here up and just say we just want to, you know, analyze our historic deal sales to date, looking at trends, conducting market research, etc.

So I was thinking, Ted, about kind of how I would use this, right, because we have not turned on this functionality in our scorecard. We're doing that right now. But let's say I'm an analyst and I support a department and I got to do a weekly report or a monthly report about how that department's doing. Could I do something where I can go and say, hey, just give me kind of a general summary of how the department's doing. So then I know where do I need to focus on? What should I bring up?

Yeah, that's a good question. So we've brought this analysis and recommendations forward to the entire organization level. And so you could take the assistant and say summarize kind of everything or summarize an exception report, just the things that are off track. And so if we do everything here, and this is actually what we do in our own organization, so we use clearpoint, we do quarterly business reviews and we use our workflows to ask everyone to make their updates. And then we just have AI produce that summary as part of the, basically the COVID page to pre reading. And then we send it out. We don't copy edit it or anything. It just, it allows our team to look carefully at it. We have all the detailed data behind it. And then once we conduct our review, we go back in and we edit this based on, you know, tangible conversations that people had. And let me just save this so we can show it on the screen and make it better throughout, right? So we can say, oh, look, there's mixed performance. And then financial. We saw different things. This particular one was just organized. This scorecard, you might remember, was kind of organized by financial, customer, internal, and then learning and growth. So the AI is going to summarize it that way. So it's just a really nice way to be able to capture all that information together. The other thing, one other thing that, that we've seen people do in Clearpoint is if you're trying to roll out OKRs across your organization or just to create alignment to your strategy across all of your departments. We can allow you to build department scorecards, create your own okrs, or if you're using a new strategic framework, something you're less familiar with, like the values discipline you can go through. And you know, we could do this for, you know, Coca Cola and start the assistant. And what it's going to do is teach you, you know, let's just say yes, it's going to be for Coke, it's going to teach you first about what the, what the organization is you're doing. So just in case there's another Coca Cola, you say no. This is for, you know, a different organization. Hopefully it doesn't get caught up here because the next thing it would do is talk to you about the values discipline, ask you to prioritize whether you're focused on, you know, customer intimacy or operational excellence. Once. I'm not going to go through the whole thing, but just to give you kind of the, the quick idea. Yeah. So it asks you, you know, your high level goals so it can take and go around innovation, customer intimacy, operational excellence, and walk you through the entire thing. So we don't need to do that for, for this, for this demo. But just to let you know, there are options here for, you know, when you're thinking about your strategy. For example, if your boss comes in and says, we're no longer going to use the balanced scorecard, we're going to use an entrepreneurial operating System or the V2BOM framework. This is a quick way to take your information and see what it would look like in those frameworks.

Sounds good. All right, so, you know, we've talked a little bit about kind of what ChatGPT and what Clearpoint can do and it's pretty obvious that it's pretty powerful and useful how it can use, but it's still a new emerging technology, so we should probably be a little mindful of some guardrails, right?

Yeah. So I'll go through one of these first ones from, from ChatGpt itself because it's the tool that we were using and you should always look at the privacy policies there. What they're telling us is they don't use the prompts that we put in it to train their model. You could imagine if, you know, everyone in high school just started filling out crazy stuff into the prompt system, it could change the outcomes of their overall model. So they're telling you that they don't do it, they're telling you that they're SOC2 compliant and that you know, you decide who has access to your information and they don't share your information. So that's what they tell you on their, their site. What do you think Rafael? Like how. What are some other ways we should, should think about this?

I'm a big believer in trust but verify. So just because OpenAI tells me something necessarily mean I trust it. So I'm very mindful of what I put in there. So part of that is don't put private information into AI. Right. Don't put confidential information in there. Don't put sensitive information in there. I work in local government, so I always use when I write an email, try to remember the use. How would that look in the newspaper kind of rule. And I take a somewhat general approach with AI so brainstorming. I'm very careful. If I'm doing a tougher brainstorm of how to deal with a difficult person, I don't include their name, things like that. You know, AI is always trying to guess to give you the best answer. Right. It's like a toddler just wants to tell you whatever it takes to get you the right answer. So understand that there's going to be some bias sometimes can hallucinate and make up stuff. So just be a little wary of using it for hiring or making assumptions about people in situations.

One of the things related to that Rafael, that, that we've done, depending on how we're using AI, I will tell it, don't make up an answer. Ask me a clarifying question and, and especially if I'm comparing it to another document or something. I could say if you don't find the exact answer in that document, tell me so that I, I know but it's still. You absolutely should be wary kind of about, about all the information you get back and, and to the extent we're using it inside of clearpoint, we will, you know, we're, we're making a human interact with the result before you hit the save button. Yep.

You know, be mindful of copyright, intellectual property rights, new technology. The courts haven't really weighed in. So you can ask it if it's copyrighted information, it's giving it to you. And you know, if you're putting in information that's copyrighted, make sure you disable chat conceptions. Just, it's the frontier. We don't really know what's going on it security. The more it knows about you, the easier it is to phish you or to do deep fakes. So don't put an entire email history or address book and you know, around transparency and contract compliance. If you're using AI to do really meaningful work, that's not an email, but helping make business big decisions. Consider informing those that you're working with on a project how you're using AI. I think depending on your role, that looks a little different. But I think especially in this day and age, it's probably a little helpful to have that conversation with people. Just be transparent. Hey, we're making a big decision. I, I used AI to help me make the decision.

Yeah, that's great. And like inside of it, you didn't see it when I share it in clearpoint, but the first time a user uses it, you do get a, a consent basically that says, hey, we're using AI with this, we save your chat log so you can see it all. But, but we do let you know. And I think it's also in thinking about, you know, some of these items, you know, there's AI in more places than just chat box like a chat GPT. You run into them in a lot of different places in tools that you use. And so that's why, you know, many times when I think about like that IT security one, right? Like, it seems obvious you wouldn't upload your email history and address book to a chat GPT when you're looking at it. But you know, when you're chatting with customer support at a different website, maybe an airline website or something, and it's kind of clear you're, you're dealing with a bot. How much information do you want to give it until you get to a live person? Right. You should never copy in your credit card information or something like that into a tool. So. So I know today we just focused on, you know, some of the open AI stuff, but there's a lot more in the AI world and just be aware of it, you know, whether you're in local government or any organization, and how you interact with it.

All right, I think that's what we have for today, right, Ted?

Yeah, we're out of time, so it was great, great chatting with you. The real Raphael, not the AI version of it. That's coming soon. Right. So greatly appreciate. Hope you have some great holidays.

All right, thank you everyone for joining us today.

All right, thanks, bye.