MSP Dumpster Fire Season 1 Episode 4: The Different types of AI

In this episode of the MSP Dumpster Fire podcast, host Alex Farling and guest Robert Chiaffi (CTO of Progressive Computing) walk listeners through the landscape of artificial intelligence, clarifying the many acronyms and categories that often confuse MSP professionals. They begin by acknowledging their own learning journeys and then systematically define and illustrate each major AI concept: - Large Language Models (LLMs) such as ChatGPT, explaining how they are trained on massive text corpora and can generate or summarize language. - Machine Learning (ML) as a sub‑field that learns patterns from data to make predictions, illustrated with speech recognition and self‑driving cars. - Deep Learning, a subset of ML that uses multi‑layer neural networks, with fraud detection as a concrete use‑case. - Neural Networks, described as software analogues of brain synapses that pass micro‑decisions through layers. - Generative AI, which creates new content (text, images, code) from prompts, with examples like DALL‑E, Midjourney, and the AI‑generated podcast artwork and music. - Natural Language Processing (NLP), enabling computers to understand spoken or written language, with applications in voice assistants, sentiment analysis, ticket triage, and sales call analytics. - Artificial Narrow Intelligence (ANI) versus Artificial General Intelligence (AGI) versus Superintelligence, outlining the scope and theoretical status of each. - Open AI models (publicly available, data‑sharing) versus Closed AI models (proprietary, data‑private), stressing security and privacy implications for MSPs. - Hallucinations, where LLMs fabricate plausible‑but‑false information, illustrated by a lawyer’s erroneous case brief, leading to a “trust but verify” policy. - AI Bias, how biased training data propagates into discriminatory outcomes, with examples from resume screening and mortgage underwriting. Throughout the conversation, the hosts emphasize practical MSP concerns: hardware acceleration enabling AI on everyday devices, cost‑effective adoption (e.g., using generative AI for graphics), and the need for policies around data handling, verification, and bias mitigation. The episode ends with a reminder to remain skeptical, verify AI outputs, and continue learning. The tone is conversational and instructional, with a supportive yet cautious sentiment toward emerging AI capabilities.
Chapters
Welcome to the MSP Dumpster Fire series. This is season 1, the impact AI will have on your MSP.
Welcome back to this week's episode of MSP Dumpster Fire. Season 1, the impact that AI will have on your MSP. My name's Alex Farling and I'm one of the co-founders over here at Empath. I'm joined by my esteemed colleague, Robert Chiaffi. Robert, go ahead and introduce yourself.
Thank you, Alex. Robert Chiaffi, the CTO and co-founder of Progressive Computing, an MSP based in Yonkers, New York that I co-founded in 1993. So we've been in business a little over 31 years now.
And Robert, we've been kind of unpacking AI. We've had, what, this is our 4th episode. So a few chances to kind of turn the screws on AI. But we got to the point where we decided we had to really unpack what all of the different kinds of AI are. So that was AIR. That's not really what the different types of artificial intelligence actually are. So that the audience really understands not all AI is created equal. Not all AI is created with the same purpose. There's a lot more to it than just lumping all AI into one bucket.
Well, you know, full disclosure for the audience, 100% what you just said is one of the things that I started to do on my learning journey, right? I'm still a student of AI. And one of the first things that I did was say, I just have to get my arms around what all these acronyms being thrown around are out there, right?, you know, GenAI and LLMs and, you know, machine learning or ML, like all of this stuff and acronyms that I was reading, I'm lost. What's the difference? Isn't AI just AI? And what I soon quickly discovered was no, no, they're very different.
Well, and that was kind of one of the things that we talked about when we agreed to start this, this series was, I mean, we've been talking about AI for, about AI for what feels like years now. And we were kind of afraid that people were just as lost as us and afraid to raise their hand and go, hey, I don't really know. So you kind of took an exploratory journey and said, I'm going to play student and I'm going to go dig in and learn this stuff. You're still kind of on the educational journey on this, right? I think you kind of tell us where you're at in that journey and then we'll start to unpack these different kinds of AI.
Great question. And, you know, I think just about everybody out there, including the so-called quote unquote experts, are still learning themselves because the, you know, the industry is moving and changing. Changing at a very, very quick pace. So if you're feeling behind, don't feel like you're at a loss or well behind everybody else, because I think most of us are sort of in that bucket. And what I did was I just went out to the internet, and I went to some of these educational platforms. I'm sorry, it wasn't Empath. You guys weren't ready yet.
I don't think we're prepared to have this conversation yet. Yeah.
But it's, you know, it's some of the names that you know, and trust that are out there. And I, you know, I probably put in about 20 hours of instructor-led, you know, video-based training, right? And, you know, there were quizzes and tests I had to take. And, and again, it doesn't make me an expert in this. It just makes me someone who just decided to inform themselves a little bit better because I felt I needed to understand enough to have or lead a conversation with my clients, as well as with my peers, and even understand what's being I'll say, fed to me from vendors on artificial intelligence so that I could process and be an intelligent participant in these conversations. And so, again, being, I guess, a computer scientist, if you would, I felt like I needed to get a good vocabulary. And that's where I focused quite a bit of my time early on in this process.
And AI is kind of every vendor and every marketer's favorite buzzword right now, right? We think if we tack AI on the end of it, we're going We're going to get a 2x higher valuation when we exit our company or whatever. But in reality, AI doesn't mean so much as some of the different kinds of AI that we're going to talk about. And this is where we get to sit down and really kind of unpack those. So let's jump in. One of the ones that we see the most, whether we're aware of it or not, is large language models, right? These are what we see from GPT and all of the big players that are out in front of the community taking off so, so quickly because they're really easy to access and easy for everybody to get ahold of. Why don't you unpack the large language model and help everybody understand what that really is?
And so the way I'm gonna unpack each one of these is I'm just gonna kind of read off a definition for you from my notes here and then maybe give or talk through some of them with a little bit of example. So large language models, LLMs, what the heck is this thing, right? It's a type of computer program that can understand and generate human language, right? LLMs are trained on massive amounts of text data. By the way, we talked about this in the last episode of training AI with large datasets. So they take large amounts of text-based data and can use that to perform a wide range of natural language processing tasks, such as translation, So if you've used a translator app from one language to another, right? I'm very proficient in Italian, but I still have a translator app that I use to remind myself about either grammar or a word that I've forgotten or don't know how to say. So such as translation, summarization, and text generation. So the best example that we have of this is ChatGPT. And another good example would be like chatbots, right? These are, really good at taking a response from a human, processing it, and then giving or mimicking a response back. And let me give you a quick example. So this is the one that I kind of just wrote up just for my own purposes of trying to reinforce concepts. So imagine ChatGPT asking you, what can I help you with? And I might prompt it. Prompting, there's another term that we've been talking about. Write me a short, cordial, but firm email to the CEO asking for a 20% raise, citing my 15 years of loyalty, consistent 97% budget attainment record, and unmatched 100% customer satisfaction rating, right? And so, think about everything that I just said. I gave it a lot of very specifics that, you know, are important, maybe even dare I say some emotional things that are important to a human. And You'd expect that ChatGPT or a Copilot or something like that would give you a pretty good response based on that. And all of that is what a large language model does. Process that input, tons of data that it's been trained on, and to give a language, a natural language response back, such as like, "This is how you'd craft an email to a CEO." And we feel like this is brand-new technology, right?
Because we're just starting to see it. Available to the masses in things like ChatGPT or Copilot or whatever. But this has really been around forever, right? If anybody's used Grammarly, um, this is a great example of it's looked at so many people's writing that it knows where a comma or a period belongs. It knows where you can remove an extra word or, um, you know, clean up your phrasing. And it's really been around with us for decades.
It has. Um, and, um, what is it also accelerated its more universal availability today is the processing capabilities of our hardware, right? If we go back 30, 40 years ago, we just didn't have the computing power to be able to do these things at least efficiently and to be able to put it in the hands of the masses. But, you know, your cell phone has enough power on it these days, processing power and memory capacity to be able to do this.
Yeah, it's amazing. That's amazing. So let's jump in into the next one in the list here. Machine learning. What is machine learning and how does that differ?
So, you know, I struggled a little bit between, you know, LLMs and machine learning. Like, what's the difference? Because, you know, and I think, and I'm just going to jump to one of the examples, speech recognition. We were just talking about how ChatGPT is processing language, right? And speech recognition, how is that different? Well, let me try to unpack it a little bit for you. So machine learning is, again, one of these subfields of AI that enables machines to learn from data and make predictions or decisions without being explicitly programmed. So the programming it gets is, I'll say, maybe the leash has been taken off and it's been given a little bit more free rein to take that data and to make some predictions or decisions based on that data instead of being confined to very explicit parameters, right? And it involves training algorithms on very large data sets. And there's the commonality in all of this, to recognize patterns and make predictions based on new data. Now, speech recognition is one of them because AI has Siri or Alexa has no idea what I'm about to say, but it has to be able to adapt in that situation to know what my intent is and what I'm asking for. But probably the best example of machine learning that I could find is self-driving cars. Think about all of the inputs that a self-driving car gets, right? You know, the automatic pilot, let's say on a Tesla, right? All of the inputs that are coming at it in random and likely in very unproven or unfamiliar spaces. You've never driven to this area of the neighborhood, the city, the state, the country, whatever. And it's just gotta on the fly take that data, learn quickly and make good decisions based on what it knows. So that's machine learning.
And that's a great example because if we think about what it takes to process an automobile rolling down the road, right? It's weather conditions, it's the kid with the bouncing basketball that pops out from behind the car. It's a construction zone where maybe you have driven this road before, but it's different today than it was yesterday. Dashed lines on one side of the road, solid lines on another side of the road, changing speed limits, all these kinds of things. That all have to be taken into consideration. And so there's all of this contextual information that the machine has to put together literally while it's rolling down the road. I mean, this is literally building the plane while it's flying down the road. It's building its plan while it rolls down the road to figure out how to behave and how to operate. And that's why this is so hard, right? It's reading from cameras and LIDAR and all kinds of craziness. To try and put together a complete picture of everything it sees around you. It's actually a little impressive that the human brain can just process all of that and function.
I mean, right, we're a little bit of diverging from getting through all the terms, but I think you hit on something really interesting. It gave me a newfound appreciation for how our minds work, right? Just our own 5 senses or sensory inputs, right? Maybe even sixth senses is time. You know, like when that Spidey sense goes off, like what sense is that? But taking all that response, and we just, the way we walk through life and just, you know, catch a falling cup or, you know, react to a situation or being able to read an expression on somebody's face and know exactly what they're thinking. That's pretty amazing considering the complexities of how we're trying to mimic that with technology.
Yeah, it's absolutely incredible. And so that kind of spells out machine learning a little bit. How is deep learning different?
Yeah, so again, it was one of these things where I said, you know, okay, deep learning, how is this different? Like, you know, I felt like, could it go yet to a different level? Well, it did, right? It's a subset. Deep learning is a subset of machine learning that uses multiple layers of something called neural networks. And I know this is getting really geeky very quickly. But it uses an interesting algorithm or programming methodology inside the software to simulate the human brain, right, and the way neural networks work to learn from very large amounts of data, but for very, very specific outcomes. And maybe the best way to explain deep learning is to give you the example which I found was fraud detection, right? Fraud detection requires a lot of sort of very nuanced, very specific discernment to understand what the difference is between me swiping my card at a Home Depot and it really is me or maybe not me, right? How does it determine those sorts of things? And you can imagine all of the inputs that go into this. But that's what deep learning is. It's this machine learning, so it can dynamically make decisions, but it's using very, very specific programming to learn from very large amounts of data so that it can have that very specific outcome.
Yeah, it's interesting to see how behavioral analytics uses different datasets and ties things together a little bit differently than just that car rolling down the road with like a machine learning or something like that. Really, really, really interesting. You used a word that some folks might not be familiar with there though. You said neural networks. Do you want to take a sidetrack for just a second and explain neural networks to folks?
Yeah. And honestly, it was a little, you know, ethereal for me to kind of like, you know, and this is coming from somebody who's trained as, you know, a computer scientist. And, but it's essentially just a way of building software that mimics the way a human brain is connected. So if anyone kind of remembers either from biology or if you watch a a lot of weird science shows like I do is, you know, the human brain has millions, billions, trillions of connections between different cells or these synapses. And it basically is each synapse makes a decision and then passes that information off to other synapses. So the way a neural network kind of works, and I'm probably oversimplifying it, and I might be butchering it a little bit, for those really hardcore computer scientists out there. But essentially what it's doing is it simulates that with software so that it can make decision. It can take a lot of input in very quickly and process it and feed its, these micro decisions to other decision trees so that it can eventually make the right decision.
Right.
So if you think of it like as a giant tree, right. With lots of branches and roots that are all like pulling in lots of information, processing it and passing it off to other roots and branches. And that's kind of what a neural network is.
That's really interesting. And I think it's helpful to help everybody really understand how some of this works and how we can process such large datasets so quickly.
Yes.
By making individual tiny decisions that lead up to the outcome that the AI is going to give us.
Correct.
So we've talked about a lot of different types of AI that are processing a ton of information. We haven't really gotten into AIs that are creating things yet, but here we go. Let's jump into generative AI.
So generative AI is the thing that a lot of people kind of see front and center, right? Because as the name implies, it generating kind of like you just said, it creates. That's, those are the tangible things that we can see very clearly. It's the type of AI that can generate either text And we gave some examples of that already. Images, code, and what I mean by code is software, right? Hey, GenAI, write me some software that does, you know, that does this specific thing, right? It can generate that source code for a program or other types of content, all just from a prompt, right? If I prompt it well enough, then generative AI can produce output for me that is good enough, close enough, or just, you know, pretty solid enough that I can then take it and do more with it. It's also used often in online tools and chatbots that allow users to type questions or instructions into an input field upon which an AI model will generate some sort of human-like response. Examples. So we talked about ChatGPT, but that's another one. DALL-E or Bard or Midjourney, any of these things that you can say, you know, draw me a picture of a cat with sunglasses wearing a hoodie who's got kind of a grimace on his, you know, a tough grimace on his face in Andy Warhol, you know, style colors, right? And it can do that. It can take that input and it can produce some output for you. Right now you may need to then tweak that a bit, but that is, that's a good example of generative AI.
And you can even do things like feed it some input, like I'm doing a podcast about this MSP dumpster fire thing that is artificial intelligence today. Give me some music to play into the podcast and a graphic to go along with it. And if you haven't caught on yet, all the graphics and music that we use to create this podcast are 100% AI-generated. So if you saw the cool logo that we've got, that is 100% AI-generated along with the music that we use to play in and play out. So as you hear those, just give— as you see and hear those, give a thought to how good or how bad that is, how far you feel like it's come. I'm super impressed by the quality of that graphic. And, you know, my assistant Francis spent a lot of time on this. He's been producing this podcast with us, and he spent a good bit of time putting that thing together. But man, it's pretty. And the, the, the cool thing, back to the whole jobs conversation that we had before, is that this couldn't have been done without a human. That took someone with, uh, with some artistic talent to be able to take what was given to him by the AI and then put the logos and the lettering and the things that we needed over top of the image to refine the image. But he gets so much brighter colors and cooler palettes and, and just different things with this generative AI than, than you could create by hand unless you're just, you know, top 2% of folks with Photoshop or whatever.
And quite frankly, Alex, like, you know, the show, like our show here, as well as the any real production of things like this, we don't have the budget to pay somebody a whole week's worth of salary to create one image, right?
Or one song.
Yeah. It enables organizations to move at a pace and do things that are, I'll even say, good enough, right? I'm sure you could have paid a graphic artist, you know, a very hefty sum of money to make a better-looking image, maybe, right?
Could have hired an orchestra to play a song, right?
But yeah, was that worth it? I mean, for this, like, probably not, right? We're not talking about something with, you know, we don't have a billion-dollar budget here.
There are a lot of places in life and in business where 80%, 90% is good enough.
Yeah.
And you get the work done and you move on. And you're right, it enables people to move way faster. And in a future episode, we'll talk to some folks who, who work alongside AI and use it in their real life. That's kind of a distraction to this conversation, but a really cool conversation.
Totally.
That we'll dive into and talk about. But taking it from generative to language processing, NLP or natural language processing is one we hear a lot. Right. Dive into that one and tell us what makes natural language processing different.
So again, there's a great example that's front and center. It's probably in most people's pockets right now, but natural language processing is the ability for computers to process human language, both written and spoken, and to understand its full meaning and complete with the speaker or writer's intent and sentiment And then finally, respond in kind, right? So the examples here are voice recognition, right? I mentioned Siri, I mentioned Alexa. Sentiment analysis, right? What's the tone? You know, is his pitch elevated? Is he angry? Is he laughing? Text summarization and language translation. So, you know, I keep using cats as an example. I love cats. Personally, so that's why I always refer to cats. But I'm not an Apple user, but I know if you ask Siri, hey Siri, do you like cats? The response is going to be, yeah, man, cats are cool. So maybe somebody—
I'm going to try that later just to make sure.
But you could go beyond that. Maybe that's a very programmed response from Siri, but there's a lot of voice interaction technologies out there like that that can understand what you're trying to get at and then can compensate to understand what it is that you're trying to say and gets the context and can respond appropriately.
Well, and we're seeing really cool uses of this kind of technology within the sales side of life and in the client success or customer service or tech support side of life, right? Gong can literally listen to your sales call. And tell you if you missed a closing opportunity because the buyer had intent in their voice or in a question that they asked or in something that they said. It can tell you who your best closers are and when the points in their conversation that are super successful are.
Yes.
Because it understands the language that it's hearing. But for the MSPs that are listening to this, we're starting to see products on the market that will read your support tickets and let you know when a user is incredibly frustrated and may need an extra holding hand. And just imagine a world where you open up a help desk ticket and there's a little stoplight that's red, yellow, green. Hey, this person's good to go, or this person's, you know, just absolutely on fire and furious with you. And you can react accordingly. Maybe tread a little lighter around the folks who have already, you know, kind of are already a little triggered or a little set off. So I think that natural language processing is an interesting one. And we see it in a lot of places that we don't really think about. About, but it's really interesting.
It's kind of hidden in the fabric of a lot of the technologies that we already use, like, know, and trust, right?
And maybe take for granted a little bit. So that's super, super interesting. Now we're getting into some that are a little more off the beaten path that I'm a little less familiar with. So now I'm gonna challenge you a little bit and see how good your homework was. Artificial narrow intelligence.
Right, which I'm just gonna kind of jump ahead to artificial general intelligence because those two sort of, compare and contrast each other. So I'm going to talk about them both at the same time. You got artificial narrow intelligence, ANI, versus artificial general intelligence, which is AGI. So the narrow intelligence is a type of AI that is designed to perform a very specific set of tasks or set of tasks such as facial recognition, speech recognition, right? We talked about in an earlier episode about how John Deere bought a company that can discern the difference between a weed and a good crop, right? Those are very, very specific things, right? Even chess programs, right? Very narrow intelligence. The general intelligence, though, contrasting to the narrow intelligence, is a type of AI that can perform an intellectual, and, you know, here's where it you got to put a little bit of a grain of salt or an asterisk on this word, can perform an intellectual task that a human can do and learn and adapt to new situations like we do. And again, it like Apple Siri is an example of that, how it can learn from the input and then make decisions. The self-driving cars, right, are things that are Even though they're doing what you would consider a specific task like driving a car, there's a lot of different inputs that it's taking and it's got to make some— it's got to control a lot of different things. It's not as narrow as you think.
Okay. So now we're going to go even further down the rabbit hole. We're into super strong artificial intelligence. What makes an artificial intelligence run around in underoos? Because I'm not sure I understand the use of— of super artificial intelligence.
Yeah, super artificial intelligence or strong artificial intelligence. I want everybody to think about all of the memes and all the sci-fi stuff that you've watched, Star Trek, Space Odyssey 2000, iRobot, WarGames, all of those things that we see in Hollywood, how Hollywood represents artificial intelligence. This is superintelligence, right? It's a form of AI that is capable of surpassing— little scary word there— surpassing human intelligence by manifesting cognitive skills and developing thinking skills on its own. One might even say, oh my goodness, this thing is like coming alive, right? Considered the most advanced and powerful and intelligent types of AI that transcends the intelligence of even the brightest human minds. Again, this is what super artificial intelligence is. However, before you all freak out and start grabbing your rifles and heading to the hillside for survival, super AI is theoretical since most practical developments are in artificial narrow intelligence. And some in that general intelligence. Some believe though that we may only be a few decades away from achieving superintelligence.
I'm just gonna reach around behind me over here and unplug HAL 9000 and we'll deal with it later. Yeah. Yeah. So concepts that have been around for a long time, but not necessarily something that we're ready for in real life.
No.
Probably requiring quantum computing and some other things that we're just technologically restricted from today, right?
Yes. Yeah.
So, so let's jump into a little bit of an explanation between the difference in types of AI between open and closed. We talk about open versus closed AI, and I think folks kind of have an understanding of that. But let's, let's really spell it out and make sure they understand.
Yeah. And I think it's, it's very important to be very specific and clear on even some of the basics, right? Especially in this business. Sometimes we tend to gloss over the basics and not understand and then get lost or tripped up, right? So this was, again, something that I learned several months ago or last— at the end of 2023 when I started diving deep into education. So open versus closed. OpenAI refers to AI models and software that are publicly available and can be modified, improved, or reused by anyone. And there's a little bit more to it than that, which I'll explain. And then closed AI refers to AI models and software that are proprietary and kept secret by their owners and developers, right? So OpenAI fosters that collaboration, innovation, and transparency to the entire community, while closed AI provides more security. And so that's why we want to do these things, security, privacy, and profitability for its owners or developers. So here's an example. If you're going to use OpenAI, and I'm naming them specifically because they're a perfect example of what we're talking about, they're actually OpenAI. Their name says it all. If you put, let's say you're an MSP and you say, "Hey, OpenAI, here's, I'm going to upload a list of 200 customers and I want you to do some analysis and la la la." whatever, you're looking, you're trying to find like what, you're looking for correlation of what companies seem to be the most profitable for you in certain geographies and types. And you're going to upload all kinds of sales information. That sounds great, right? You want AI to do that analytics for you so that you don't have to, except that you just used an OpenAI system. So the data that you just fed it has now become part of the public domain. And guess what? You can't get it back. You can't go to them and say, could you please delete that request that I just made? That's the point of OpenAI, is it takes your requests and it learns from them over time. It's taking millions of requests all day long and building and building and building that massive database that it uses to be able to respond appropriately. You're going to want to use, in that case, a closed AI system like a Copilot, right? If you're a Microsoft 365 subscriber, where that data remains private within your tenant and that it is not exposed openly to the community, right? So companies have to be really, really careful about— and I know it might sound basic to people, but some people may go, "Oh, crap. I really didn't understand the difference there." And, you know, "Geez, you know, that's something I can help my customers avoid. Avoid using OpenAI." unless it's just experimental and you don't care about the results or sharing that information from the world, and stuff that's closed. Here's a great example. Like some of us think about email. The general rule that I have is because I embarrassed myself 30 years ago on an email system by emailing somebody something. And I developed this rule. Anytime I email something, I pretend that the entire world will get a copy of it and read it. So, I keep things, private things, confidential stuff as much as I can out of email. Right now, we still do it anyway. But that's one of my kind of pressure test rules is don't do that.
Sure.
I'm going to use the more closed structure, like let's say an in-person conversation to communicate very sensitive information. So that's the differences between open and closed, and we just have to be really mindful of security and privacy and confidentiality.
And when we used to talk about these things at my MSP, we would talk about, hey, Mr. Customer, would you be okay with that information being posted on a billboard down on the highway?
Right.
Right. And when they go, oh, well, no, well, then we shouldn't be sending it in email. We shouldn't be pasting it into an open AI model because eventually that AI model will get breached. And all of that data will get made public, right? Indeed. There, there is a, a big disadvantage to how some of those are built. And the problem is also that, that AI is popping up everywhere. So it's not just OpenAI that folks have to worry about. It's the AI that's built into Notion or Evernote or name your tool.
Yeah.
They've all got one. And if you start pasting your data into that AI and having it process it, the same kind of thing is probably happening.
Yeah, be careful. Make sure you read the, you know, the disclaimers or whatever the legal language is behind those. When you click that, I agree, I accept, right? Which we all blindly do. When there's AI-powered stuff, you probably want to just pause a little bit and make sure you understand.
You're probably contributing to the model.
Yep.
Which puts your data at risk.
Listen, which is important.
Yeah.
It's important that we're contributing. Because it makes the tools and the technology better for us.
We just probably don't want to paste the recipe for Coca-Cola in there.
Exactly. Exactly.
Yeah. Yeah. So, you know, we just have to, we have to operate eyes wide open and aware. And that's where understanding what's in that user agreement, that end user license agreement or whatever, terms of service is super important. So let's talk about everybody's favorite thing. Every now and again, AI gets a little nutty. And it hallucinates, right? AI has some misconception about reality and it delivers you something that's not fully thought out or maybe not factual. And this is why we have to trust but verify our AI tools. What do we know about hallucination, Rob?
Well, so hallucination is this, and I'm going to kind of read off a definition for you. It's this phenomenon where a large language model, generative AI tool, such as a chatbot, perceives patterns that are nonexistent or imperceptible to human observers, and therefore then takes that misleading, self-misleading information and creates outputs that are nonsensical or altogether inaccurate. Now, there's a famous case about how a lawyer, like, this is very early on in ChatGPT.
I saw this.
Did some research on some casework research, right? I'm not an attorney. Certainly have plenty of clients that are, so I might get the attorney lingo a little goofy here. Forgive me, don't sue me. But he went off and did his case research and asked ChatGPT for all this information, got it, and submitted it to the court as his official brief, right? You know, when they— when the court looked at this and then went to go verify it, I don't remember if it was the opposing side or if it was the actual court itself, but they, they found that like the cases that he was citing didn't even exist, right? ChatGPT filled in the blanks based upon his prompting because it was trying to give the guy the information, the lawyer the information that he was looking for, but it had to sort of fill in the blanks. When ChatGPT and its dataset doesn't fully fill in every little crack, it starts to use like averaging algorithms or will start to kind of piece together what it, you know, by making data up to fill in gaps in its dataset. So you gotta be really careful about the output that you get, right? And that drove me to create a policy internally here, which my team is, well aware of that, and you said it already, Alex, trust but verify. You're allowed to use these tools, but understand that you, A, if you do something to use these tools, you need to disclose that you used it, and you also need to verify that the data that you're getting out of it is actually factually accurate.
Well, these tools are super powerful. You know, at Empath, we're creating courses and working on education and training for MSPs. And we're not the expert in everything that we're educating and training on. So maybe we're looking for an expert to come in and train on a topic. If we were to write kind of a real high-level, hey, we're looking for somebody to train on insert soft skill here, we can go to ChatGPT and say, hey, what would a good course outline look like for XYZ soft skill? And it can fill in a lot of the blanks for us. Now we want to read that and sanity check and go, oh, it thought of some things I wouldn't have. Oh, this one's weird. Maybe it doesn't fit in. And strip it out of there. And that's kind of how you have to treat these things, right? We have to know that sometimes they're programmed to be a little too eager to please, and they will give us information that is not exactly what we're looking for. The human touch is always going to improve the output that comes from the AI. Awesome. So this is going to kind of lead into the right into the next one, which is AI bias. I'm sure we have some some amazing examples of AI bias, but let's define it first.
Well, I've got one good example that I'll talk about because I think it's probably the easiest thing to talk about. And, you know, if you've watched the news, especially AI-related news, you probably have seen this, but AI bias refers to the occurrence of biased results due to human biases that skew the original training data or AI algorithm. Right? And this can lead to distorted outputs or potentially harmful outcomes. Now, again, let's rewind the conversation back to training, right? AI is some cool software programming, but it's also data. And it's data that a data scientist has to say, this is what a good example of— this dataset is good and this dataset is not good. Now imagine that that dataset that you're looking at is, let's say, resumes, right? And there's the example, resume screening. So the data scientist working on training the system on what good data, what good resumes look like versus what bad resumes look like. Well, let's just imagine that data scientist had a college rivalry with, you know, some other school and marked all the people with an educational background in that rival school as bad, right? And so now bias has now entered into the resume screening tool, right? Now, this could be along racial lines. It could be along, you know, male/female. It could be biases about like different types of experience or lack of experience. I'll throw one out there. I absolutely love retail experience.
Why?
Because you're dealing with the angry public and hopefully you're dealing also with money, especially cashiers, right? I like people who know how to count back money. To me, this builds character. So as an employer, I love to see that. Now, will I hire you just on that alone? Probably not. But that's my bias, right? And if I was a data scientist and I was training the set, oh, look, someone worked in a supermarket, good resume, right? Maybe the rest of their resume is awful. So I can get bad results from this, right? And then if you think about now I've hired people that are performing jobs that may not that might lead to some catastrophes, like there might be an engineer and there might be a flaw in their ability to design something that's structurally sound, or they might be working in a medical profession and may not necessarily dispense either the right medication or the right advice. So there's, you know, that's the downstream sort of effects of AI bias. But the right front and center is how it affects humans who are, you know, imagine a mortgage processing AI tool, right, to screen out mortgages and a subset of people who might feel, and I don't mean a subset of people, but I mean a group of people who might feel discriminated against because maybe the data scientist who trained that system did a poor job. And guess what, by the way, and I don't even mean to say that it was intentional, There could be unintentional biases, right? You just don't understand. You didn't realize that you're training it with a bias of your experience. And maybe I'll use right there the example of how I love retail experience, right? I would— oh, what, you know, that's just harmless. Like, that's just, you know, well, there's a bias there, right? And you don't realize that it's going to have an impact.
And we see bias all over in our lives, right? But if you think about the fact that we're training AI AI on information that already exists in the wild. We're training AI on information that exists in the real world. Well, if you pick any 2 or 3 news sources and look up the same event, you're going to get very different recaps of what happened at a historical event. Totally.
Yep.
These are examples of bias that live in the mainstream that if someone were to take those articles as fact and feed them into an AI, that AI is getting ingested by us totally on accident or totally unintended, I should say.
And that's the scary thing, Alex, what you're talking about now. I mean, I don't mean to get political or get into our social issues here in this country, but I think it's important that we talk about—
Our social issues feed AI.
Yeah, well, that's what I'm kind of getting at. Imagine somebody creates a news media outlet that uses AI to to pull in social media feeds to build its news stories, and then it's just, it becomes an amplification or an echo chamber, right? There's a lot of danger in this stuff, right? And suddenly you got a bunch of people who are being convinced of certain facts that are actually not true, right? Right.
And the AI will straight-faced lie to you about hallucinations.
And there's a NASA scientist that I follow. She's brilliant. She's really, really good. And she actually— I was watching a little clip of her from her this morning. And she was talking about all these articles out there about a comet that's heading straight towards Earth, right? And this comet is like 31 kilometers wide. And all the articles that she's citing all have that headline, you know, dangerous comet on its way heading towards Earth. It is heading towards Earth. In astronomical units, it's heading towards Earth, meaning it's going to miss us by 4 times the distance from the Earth to the sun. So it's going to, like, by our standards, it's going to widely past us. But in the context of astronomical units, planetary and solar systems—
It's in our general direction.
It looks like it's like you can't even figure out that it missed us, right? But it's coming in our— And so that's what I'm kind of getting at is people pick up on things. And then they start to feed this frenzy without like, did you look at the nuanced facts to understand that this thing actually isn't going to slam into the earth?
So I think again, we circle right back to where we started, which is trust but verify and the importance of understanding what your AI is telling you. So I think that kind of puts a good ribbon on this topic.
It does.
Let's tie this one up. I have some work to do now. I've decided. While listening to you talk about this, I'm going to go download the entire Babylon Bee and feed it to an AI, see what I can come up with.
Don't do that.
It could be fun. It could be fun.
Better things to do.
You should take way better things to do.
Take the courses that I took.
There you go. There you go. Rob, you're taking all the fun out of my life. But guys, I think that does it for episode 4. Thanks for joining us. And we'll be back in a couple of days with episode 5.
Peace.
Thanks. This podcast was created by me, Alex Farling, and my good friend Robert Joffe. This podcast is produced by Francis Bellin. The music and artwork for this series were produced by artificial intelligence and edited by Francis Bellin.










