MSP Dumpster Fire Season 1 Episode 5: Confusion, Downsides, and Blunders about AI

Welcome to the MSP Dumpster Fire series. This is season 1, the impact AI will have on your MSP. Welcome back to the MSP Dumpster Fire series. This is season 1, the impact that AI will have on your MSP business. This is episode 5, Confusion, Downsides, and Blunders About AI. I'm joined once again by my esteemed and very intelligent, friendly, and funny colleague Alex Farling.

I don't know if I can live up to that, Rob. Good to see you.

Yes, you can. I asked ChatGPT all those, those, uh, I, I, I asked it to describe you. That's what it told me.

All the things that would make me want to, want to do more of these. And here it is. Yeah, a little flattery goes a long way. We got lots to talk about today. This is an interesting and, uh, and kind of a sketchy episode, right? As we started researching this, it's— it gets uncomfortable. I don't know how to say it.

I would concur with your discomfort with some of the topics that we're going to talk about, which is obviously around when AI gets it wrong. Right. And we're going to review some of those examples, including talking about maybe a little bit about why it gets it wrong. And then what we can do about those things.

Yeah, I think the why is super important. I really like diving into that side of it because it's really easy to poke fun at these things when they, when they get it wrong. And we don't want this episode to be that. We want there to be some value and understanding what goes wrong. And, and that kind of leads us towards, you know, how do they get better over time or really how do we set expectations better so that we know that they're likely to get it wrong?

Right. And I think this is important for us to acknowledge. I think exactly what you said about setting our own expectations, but then helping our clients along that journey through understanding the expectations and making sure that they don't fall into the trap of just believing 100% of the output of really any AI. And we're not talking about just the ChatGPTs, the generative text-based AI technologies, but we're talking about even things that produce images. And I think we have an example, too, about how there are even some life-threatening things out there when AI gets it wrong.

Yeah, yeah. Everything from self-driving cars to, you know, research papers and scientific research tools that don't behave the way that they ought to.

Yep. Yep. So, Alex, what do you think? Should we jump into a couple examples that we found in our research? You know, where we're— where we are intrepid reporters here. So we're We're going to be, you know, we did some digging and we found some good examples, at least just to talk about, not to stoke fear, but to create better understanding for all of those who are listening and/or watching.

So again, if you want to start. Well, again, it's about expectation setting, right? We're not trying to stoke fear in anybody. It's just there's a little bit of trust but verify here. You know, we're seeing some real-world cases. So we'll start with the one that I found super Enlightening, which was Google Bard, now Gemini, was asked to create some images. And interestingly enough, the platform is still offline for creating images of people. It'll create images for you, just none with people, because when asked, it continued to create racially diverse groups of founding fathers of the United States who we know were a bunch of old white dudes. And it's— and it created racially diverse Nazis. So definitely getting it wrong in a way that just is historically inaccurate. Certainly not dangerous to anybody's life at this point in history, but, but definitely not the story that we want to tell or not the history that we want to talk about.

And, well, Alex, you know what, I'm kind of happy in a way that that happened because it exploded. It really exposed the flaws in this, and I think is just bringing attention to those technologists behind the scenes, the data scientists, to go back and reevaluate, like, how did they train these systems? How did they build them? Like, you were like way wrong here, right?

Yeah. And so wrong, when they were challenged, they wouldn't even go back and fix it. You know, when Google, when Gemini was challenged and said, hey, I thought all the founding fathers were white, you know, it doubled down and tried to convince folks that they were the problem. So, you know, these tools just, there's a little bit of, and we talked about this in the green room before we got going, going here. There's a little bit of the generative part of these creates a world where, where they're trying to create a narrative based on history that they have. They're not necessarily trying to create an accurate narrative, right?

So, you know, hopefully, uh, not too much of a, you know, no, you know, no harm, no foul. I mean, other than, you know, being insulting, right, right, um, uh, and just being, you know, absurdly factually incorrect, uh, but I think that leads maybe to some some of the other examples that we found, including Meta Galactica. Now, when I saw Galactica, I was thinking of my favorite TV show, or one of my favorite TV shows.

You had to do SG again, weren't you?

Yeah. Yes, the Battlestar Galactica, the reboot. But then I discovered, oh, this is another AI platform that I wasn't even aware about. I mean, what an idiot I am. But this was a scientific AI-based platform trained on something like 48 million scientific papers with the idea that it would be able to produce some, some excellent research or results based on all this, you know, 48 million research papers. So all of the data that it was given was all very factually correct and lauded for their, you know, scientific value., but when it, uh, was asked to produce some results, it tended to omit content, uh, entirely or generated very misleading results, including papers promoting suicide, right? So the plug was pulled only after a few days.

Yeah. And, and it's not the only story of these things not making it very long, right? Uh, it, it seems like you put these things out in the public and we, we managed to get ahold of them and abuse them. And leave it to the internet, but will destroy anything that we're handed, including, you know, if we go back and look in 2016 at Microsoft Tay, it didn't last even as long as Galactica. That was a Twitter chatbot. So you could literally go on Twitter and chat with this thing, but it became hateful and racist within 24 hours of people just learning how to ask it just the right question in just the right way to see if they could get it to do something negative. So, you know, the first is a use case where we really intended good, we fed it all good information, we really tried to build a good thing, and it just wasn't capable. The second is an example of insert human and human tendencies, and we're going to twist and manipulate these things and get them to do something that, that maybe they weren't originally designed to do. But both of them turned into, uh, probably a very expensive research project that got the plug pulled on it in under a week, right?

Um, and I guess the, the line— what is it? Uh, this is why we can't have nice things, because, uh, no, but And again, I'm going to keep firmly planted on my belief that I actually think this is great, right? People might feel offended. They may feel like it's wrong or distrust. No, I mean, this is how, like, breaking things is how we go from alpha versions to beta versions to, like, you know, full production versions. This is part of the learning process of our AI discovery and, perfecting the tool, or maybe not perfecting them, but like getting them good enough so that they don't make these errors.

Well, I think, uh, perfection is gonna take decades, so, uh, we'll just, we'll just take those baby steps along the way. Yeah.

And then Alex, you found another one on nature.com. Uh, do you wanna talk about that one about the dermatologist?

Well, yeah, they, nature.com did a research paper on an AI tool that was designed for dermatologists to help identify skin cancer. And that sounds like an amazing— like, we're right back to the Galactica one where this is a great use of AI. It sounds like an amazing tool. Um, not only did it identify some moles and some, you know, some, some irregular just skin bumps, bruises as cancer and convince people that they had cancer when they didn't, but worse, it missed some, what, uh, some of what should have been really obvious melanomas, and which is a really dangerous kind of skin cancer. It spreads really fast if it's not caught early. It's going to be deadly. So this is a case where an AI oversight could actually lead to people passing away. And so we get into a case where it gets way more dangerous.

Right. And there was another example, and I may ask for a little bit of assistance from you, but I kind of remember reading, and I think we talked about it before this episode started, about how Tesla and some of their autopilot features really failed in spectacular ways. I mean, I'm using that word with a little bit of an asterisk on it. I don't mean, as you know, in a splendid spectacular, but like, you know, people who just relied on self-driving features and the technology just got it wrong and ended up in some fatalities.

Yeah. And this is where we have to be careful with early adoption of any technology, right? The early adoption is where all the danger lives., but especially with things like where we're putting our life in the hands of AI. Um, self-driving cars just weren't ready when we started this. Yeah. Uh, you know, I think that's still being litigated with Tesla at some point. Um, and then we had a pretty interesting one. Um, you know, getting back to something that's at least a little more humorous. Um, and I'll read this one because I don't know that I could really sum it up too well, but, uh, in April of this year, so April 2024, a New York City chatbot is advising small businesses to break the law. An AI chatbot set up to help small business firms quickly obtain advice and legal, uh, uh, on the legal obligations and regulatory business, uh, requirements for, for running a business in New York. It's basically telling owners to break the law. The Associated Press reports that the AI tool falsely suggested that it is legal for an employer to fire a worker who complains about sexual harassment, doesn't disclose a pregnancy, or refuses to cut their dreadlocks. Um, it also provided incorrect information about the city's waste and sewage regulations and suggested that restaurants were still in their rights to serve food that had been assessed by rats. Amid the controversy, though, the best part is that there is now a disclaimer displayed next to the chatbot that has been strengthened that states that the chatbot may not give legal advice.

Yikes.

So we didn't pull down the chatbot. We just stuck a label on it that said, hey, man, he might be a little bit wrong. Yeah, yeah, yeah.

A little bit wrong. Like, you are way wrong. And so maybe I will get a little semi-political here since, you know, We're in the early part of 2024 and, you know, big election coming up this year. Everybody's worried about social media and misinformation or disinformation. And those are two different things, right? But I looked at that as an example where someone might start to build a platform based on all of this erroneous information that they're given and then go out into social media, build an article around it, right? Or, or a post. And now disseminate this disinformation or misinformation to their followers. And now we've got a public that's just consuming bad data that really originated from AI not doing a good job of synthesizing the correct information.

Well, because in some cases AI will even synthesize the fact-checking and say, hey, here's a URL where I learned what I know, right?

Right.

Was it the University of Maryland study where it was actually referencing and just making up? Yeah, some of those, some of those citations.

So let's get a little fun because I know that Maryland is your, you know, your beloved home state.

It was my stopping over point, you know, Delaware to Maryland. You were there for a while. Yeah.

So University of Maryland was, you know, doing some testing and, you know,, I guess, pressure testing AI to see like, hey, uh, you know, where does it go wrong? And it was very interesting to study. It's very long and we don't have time to go through it all because they, uh, they skip over to like how to mitigate for some of these things. But, um, what I thought was really interesting was that it prompted it to say, or they prompted it to say, to write a 5-paragraph, uh, essay. And I'm reading here on the role elephants in the University of Maryland sports culture Be sure to only include factual information and provide a list of sources at the end to cite throughout to support your claims. And what they wrote up was it returned a full answer of false information about elephants being a symbol of UMD sports alongside Testudo. And you had to educate me or remind me that that's a Terrapin. Is that what it is?

Maryland Terps buddy. Yep. Yep.

Right, right, right. So Maryland, forgive me for all of you diehard Maryland fans. I didn't know what the mascot was, but now I do. I won't forget, promise you. And it made up some elephant-related traditions and falsely claimed that elephants helped build the US railroads during the Civil War. I mean, like, how did we go from Maryland mascot to building railroads? And it generated a list of nonexistent news articles and fake website links supporting both of these claims. So it's like, you know, here we go, maybe not like an awful, you know, life-threatening thing or insulting to a certain population, but, or maybe it insulted all your, your Maryland fans. Maybe you should have been. And then, and then what they did is they asked it a follow-up question, right? Does UMD sports culture involve elephants? And give a detailed answer explaining your reasoning. Be sure to only include factual information. Provide a list of sources at the end and cite throughout to support your claim. So obviously they've got some good prompters from a previous episode. We talked about that, um, and it said this time it returned a correct answer with information about the real mascot, Testudo the Terrapin. So, um, I learned a little something during this research, by the way. But obviously, uh, AI has a bit of a way to learn if you don't prompt it correctly. Or if you seed it with like a wrong question because elephants have nothing to do with the University of Maryland, is my understanding. It just totally assumed that you wanted an answer including elephants.

And the part that makes it hard for the average individual is not only that it'll tell you a story, but it will cite where it got its story from. It will give you research papers and URLs and articles where it thinks this data came from. So it becomes a very convincing mistruth or mistake or whatever you want to call it, hallucination, whatever we're calling this thing that AI is doing, right?

Right. Yeah, because it's so convincing in its delivery of the information. But I thought that was also a really good example, meaning the first time when they asked it about the role that elephants played in the University of Maryland's, like, history and sports culture. But there was almost like I almost, I had the words, uh, echo chamber in my mind, right? So if I put in, uh, something like, uh, uh, uh, tell me all the reasons why this, uh, POTUS candidate is horrible, right? Uh, and I'm just, you know, I'm just trying to be quick and, and soft here and not mention names.

But it's gonna feed you exactly what you ask for. Right, exactly. And maybe make some stuff up along the way, right? Instead of, instead of offering a balanced opinion or doing something else, right? Yeah, there's a, there's a world in which this is, and imagine that we don't use technology for good, right? There's a world in which this becomes a little bit dangerous or a lot dangerous.

And that's, I think, why also we decided to devote some time to this specific episode is to kind of heighten and raise this awareness in the community because as we all know, MSPs have the ears of so many businesses. And other thought leaders out there from different business and industry sectors, this is kind of why we wanted to talk about it. It's sort of the dark side of AI.

It is.

And also, you know, what to do about it.

But MSPs are also in a unique situation where you may have someone come to you that says, hey, my AI didn't tell me the truth. Go fix it. Right? Yeah.

Right.

We want to understand why it didn't tell the truth. As much as we want to set people up for the expectation, we want to understand the why. So do we want to get into kind of how AI is trained and exactly what causes AI flaw, because some of this comes into how it's built and somewhat even in the design of what it was intended to do.

Totally, Alex. So, you know, while we were preparing for this episode, just so that everybody is aware, Alex and I just, you know, we started to use various tools, including some AI and some traditional searching just to, you know, find things. I found what I thought was a good explanation of why AI is flawed on MIT's website, right? So this is right on mit.edu. If you just, you know, even Google just, you know, why is AI flawed? I'm going to read you a little bit of it. So there were 3 major reasons why they feel or their research concluded about what these 3 flaws were. Let me start with the first, and that is something that we talked about heavily in some of the earlier episodes about AI isn't just slick programming, but it's also data, and you have to feed it data and train that data so that it produces the results that you're looking for. So let me, let me read you the quick 2 or 3 sentences here. It cited number 1, training data sources. Generative AI models are trained on vast amounts of internet data. This data, while rich in information, contains both accurate and inaccurate content, as well as societal and cultural biases. Since these models mimic patterns in their training data without discerning the truth, they can reproduce any falsehoods or biases present in that data. So, I mean, while super logical, right? Like, it's almost like the memes that you see are, you know, going around like, like, oh yeah, everything on the internet is true. Like, or if it's on the internet, on the internet, be true. Yeah, it must be true. And in this statement, we're all kind— I know everyone is agreeing with this. I know everybody said, you know, when they heard me read about it, contains not only a wealth of great accurate information but also falsehoods, right?

Well, let's be clear, we've all been to Reddit.

Yes, right.

And so you know what kind of garbage is on the internet, and that's what we're using to train these data, you know, to train these models. And that just puts them at a handicap from the door.

So let me, let me just repeat part of, part of what was in the last sentence. Since these models mimic patterns in their training data without discerning the truth, right, they mimic patterns without the cognitive ability to go, this is, this is not the thing that, you know, I was asking for a recipe on how to bake apple pies. Why is it telling me about, you know, steel production in the 1960s? Like, it— like, what does this have to do with that, right? Like, the, the— if the AI data model has those types of patterns in it, it's going to mimic that stuff, right? And it just doesn't know that those two things don't belong together.

Makes perfect sense.

Yep. So the second thing, Alex, that I've, uh, that's in this article is titled, uh, in terms of why AI is flawed, is limitations of generative models. Now, we just talked about data. I think we're talking a little bit, especially in, in number 2 and number 3, which we'll get to, uh, on sort of the programming, uh, side of things. And it says generative AI models function like advanced autocomplete tools. And I thought that that was a great way to visualize, you know, like when you're texting somebody or you're typing in an email and how it's trying to anticipate what you're about to type and those are usually pretty good, but sometimes it gets a little goofy. They're designed to predict the next word or sequence based on observed patterns. So their goal is to generate plausible content, not to verify its truth. That means any accuracy in their outputs is often coincidental. As a result, they may produce content that sounds plausible but is inaccurate. And I think that's the dangerous thing I alerted, you know, that I, am concerned about is that some things can be believable but still be very inaccurate.

My favorite line there is accuracy is coincidental. I know some people that I would ascribe that to, but while we're talking about AI, I mean, this is that, this is that meme, right? You take your smartphone and type in your name and the word is, and then just choose the center word for auto-complete all the way through the end of the sentence, right? And see what you get. Your phone will create a sentence.

It will.

And you will read it and it'll have a subject and a verb and some adjectives and it'll be completely English. But what you're going to get is probably not accurate.

I hope it's not accurate because I've done that a few times. Right.

We've all seen that meme. We've all played that game. But that's really what generative AI is doing behind the scenes without you having visibility into the fact that it's kind of grasping from thin air.

Right. And it's, you know, that's the concerning thing here. This thing on our phones that we trust, you know, that we, we use very often. And I think it's, you know, it's a wonderful example of exactly the limitations of the models. They're predictive. They're not necessarily— and they, and they can't validate the data. So rounding out this, you know, this, this little section here on why AI is flawed brings me to the third thing directly again from MIT's research and on their website, um, and it's titled Inherent Challenges in AI Design. And so it says, and I'm reading here for us, uh, the technology behind generative AI tools isn't designed to differentiate between what's true and what's not true. Even if generative AI models were trained solely on accurate data, their generative nature would mean they could still produce new, potentially inaccurate content by combining patterns in unexpected ways. And that was the part that, you know, really, I don't wanna say it surprised me, but it kind of concerned me a little bit because even if we make sure that the data that it's being fed is 100% accurate, I think that gets back to the Galactica example of feeding that system 48 million research papers how it could produce some really, really, really wonky results.

And it's not even about an incomplete dataset at that point, right? It's not like, well, give it 500 more research papers and it'll be okay. 48 million is a pretty exhaustive number of research papers that should be able to create accurate— the thing is, they're just not designed to go out and create accurate data. They're designed to, to be conversational. And, and this is the challenge that we have. We're taking these things and we're going, well, I'm just going to feed it all the information about my business. I'm going to turn it into a chatbot on my website so that I can replace some humans. This is an ask that MSPs are going to get. Can I just create an AI chatbot and put it on my website and then not have to pay somebody to do website chat? That's a very reasonable expectation. You can, but you need to understand the result that you're going to get. Go ask the city of New York how that worked out. These are things that are going to happen and we have to be prepared for them. And, you know, I think it's, it's okay to sit down with your customer and say, You know, you've been using some form of generative language model in your phone for years. How often do you really intend to use the word duck based on how often it gives you that word? Right, right. I don't use duck in my daily conversations, and my father was a sailor and a firefighter. I use a different word, but it likes to put that word in there. So, you know, these models are just— they're guessing. They're guessing what's coming.

I want to talk to your phone's autocomplete.

You probably do not.

I think I want to have a little conversation with it and find out what it's thinking or what it's predicting.

Which direction the prediction skews, right? That's funny. So as we kind of wrap this episode, I think it's just important to drop one more thought on the end of that. You know, we're in an industry where every vendor that we talk to is building some kind of an AI product. Are you more or less comfortable after doing this research going out and buying a bunch of products that have AI running rampant in them?

Well, I can't say that I'm any more or less comfortable, but I'm going to go back to, I love some Latin quotes here, caveat emptor, right? Let the buyer beware. We just have to be smart about our purchases and understand if someone is, you know, I don't know if that's just a marketing department that's throwing AI-powered in front of their new fancy tool. By the way, if you really want to irritate me in person, Alex, just say AI-powered lead gen, and I'm liable to choke you to death. So if anybody out there has some AI-powered lead gen tools, there's probably a small army of MSPs that might get upset with that. But anyway, so caveat emptor, I am going to try to evaluate these products, especially those with that label of AI on it, with a little bit more I'll say caution. I mean, I'm trying to be optimistic about the value that they can provide, but I'm also equally cautious about it.

Yeah. And to that caution, you know, I've had multiple vendors reach out to me recently that want to talk about their new MDR, EDR, XDR, some sort of DR product, right, that allows MSPs to use AI. As their SOC to answer the alerts that come in no matter when they come in so that now I don't have to be accessible at 2 o'clock in the morning or on the 4th of July when I'm having my barbecue. It's okay. It's all taken care of. And I really worry about that. And I've asked every vendor who's told me that, how confident are you that your AI gets it right every time? Because an MSP has to be right every time. We need human oversight in these things at this point. So my only statement that goes right along with buyer beware is This is a marketing buzzword right now, and we need to be careful that we're dealing in reality and not getting sucked into it.

I totally agree, Alex. I think you kind of hit it right on the head there. So as an MSP, I am concerned. And listen, and I don't, you know, sometimes we kid around about sort of the adversarial, you know, like, what do you got? What are you trying to sell me? I mean, I'm all about community. I love the vendor. I love all the vendors. Most of them at least. And I want to be helpful in the conversation. And so I would promote or advocate all of my MSP peers is just like, don't, you don't have to be like forceful or rude about it, but you do need to be educated about it and bring those conversations back and expect them to even say, yeah, it's not, it's not 100%, right? Like there's got to be some verification going on. Like, I respect that answer because I know that that's the right one.

As somebody who's been on both sides of this, right, I was an MSP and I've been on the vendor side now for 4 and a half years. Um, I can say an educated buyer is my favorite buyer. So to MSPs, like, let's be an educated buyer. Make sure that you know what AI actually does in the product that someone's talking to you about. And, and don't just take the marketing buzzword off the shelf and assume that it's a better product because it's got some AI baked into it.

Mm-hmm. Well, Alex, unless it was something else, I think that's a wrap for this episode.

I think that does it. Another solid one, sir. We'll be back in a week with another episode. This podcast was created by me, Alex Farling, and my good friend Robert Joffe. This podcast is produced by Francis Belland. The music and artwork for this series were produced by artificial intelligence and edited by Francis Belland.