In this webinar, Colin Brown addresses the question of which AI chatbot to use by first listing the most popular large‑language‑model (LLM) chat services—ChatGPT (OpenAI), Microsoft Copilot, Claude (Anthropic), Perplexity, Gemini (Google), Llama (Meta), Grok (X), and Deepseek. He demonstrates a systematic comparison: using a single prompt from NACVA’s AI data library, he pastes it into each chatbot’s free tier (except ChatGPT‑5, which offers the same results on its free version) and records the outputs. For each model he notes how the response is structured, the depth of research, inclusion of sources, and the specific product ideas generated for a manufacturing data‑prep problem. Grok provides numbered sections but mixes ideas; ChatGPT‑5 offers well‑organized sections with references and pricing estimates; Gemini delivers an executive‑summary style with detailed research links and code suggestions; Claude returns fast, well‑structured ideas with unique concepts like predictive‑maintenance alerts; Microsoft Copilot, built on ChatGPT‑5, yields yet another set of ideas. Colin concludes that each chatbot produces distinct, valuable ideas, so the best practice is to run the same prompt across multiple models and then aggregate the results for further evaluation. He suggests feeding the combined ideas back into a chatbot to rank them for promise, and encourages viewers to try the exercise with their own business prompts.
Chapters
On a recent webinar, someone asked the question, well, how do I know which AI chatbot I should be using? That is a complicated question, and I'm going to try and break it down here in this session. So the first thing you want to do is know the most popular chatbots that exist today. Those would be ChatGPT, which we've all heard of, that's from OpenAI. You've got Microsoft Copilot from Microsoft, Claude from Anthropic, Perplexity, Gemini from Google, Llama from Meta, Facebook. Got Grok, and that is from X. I have Deepseek here at the bottom. It is a China-based chatbot, has good results. It's free and open source, but you'll want to use that at your own discretion. So the first thing I'd want to do to test these out and figure out which one is the best— it's not so much which one is the best, but which one is the best for what you're trying to accomplish. What I'm going to do though, just to show the difference between all of these. And there are about a couple million LLMs out there. An LLM is a large language model, which is what popular AI chats are built upon. But what I'm gonna do is go over to NACVA's AI data library, and I am going to pick a prompt and I'm going to copy that prompt. I'm gonna paste that into each of these top or more popular AI chatbots and see what kind of results we get. So let's start off with Grok from X. All I do is copy. And by the way, I'm going to be using the free version of all of these products except on ChatGPT. However, they, with ChatGPT-5, they have made it available on the free version. So you'll basically get the same results. So I've just copied and pasted that prompt. Let that run and let's see what we get. Okay, as we review the results from Groq, each of these sections, they've been numbered. You've got a pain point, target audience, which is what we've asked it to do, why it hurts, the tool idea. In this case, it's coming up with a web-based data uploader that uses simple AI to auto-categorize, clean, and structure manufacturing data. Interesting. Now what's interesting here also is we're now going to change. It's not very obvious that we've changed. We've got here to the bonus and then it's changed to the, a different idea, but it doesn't really specify that. So now this next tool idea is an AI-powered cost estimator app where users input parts specs. So that's an interesting idea as well. But once again, the thing I don't like about this is we get down to the bonus here and it comes up with a new idea, but it's not obvious. It's just putting these all just in a giant list. Let's go to ChatGPT, find out what it says. We're using ChatGPT-5. I'm going to post that in there. I like in the new version of ChatGPT, it's also giving references to where it finds data. In this case, it's doing research for us and finding what other people are specifying or pain points for their business. Now this one is better organized because it's showing an AI backend complexity and fragmentation. That's the, that's the problem. And you can see it gives you the tool idea, which is an AI backend starter kit deployment orchestrator. Very good idea as well. And you can see it's just organized better. So now this is number 2, but it puts all of the different segments in an organized fashion. You can see easily what the tool idea is. It doesn't run together. And let's look at this idea, a lightweight data prep wizard for sensor production data. Interesting. It's also a really good idea. Dirty, unstructured manufacturing data, poor data readiness. So it's telling us what the problem is, how we're gonna fix it. And the tool idea is to create an edge-to-cloud hybrid data pipeline using free tier cloud. Okay, that's a good idea as well. I might have to use some of these ideas for some new products that I create. It also came up with automatically this summary table. So you can see the pain point, tool idea, how much you could probably charge for that. So the results on that are very good. The ideas are really good, and the organization of the data as it's been returned is also excellent. Let's take that same thing and put it into Gemini. One thing I really like about Gemini is how it goes through and shows you what it's doing. It's showing you this deconstructing the problem, doing some strategic research, creating a plan to do that. And then it actually shows the research that is being done on these websites. So this is where it's going out to find people who are expressing problems that they might be having in their business. It synthesizes the core problems, addresses a major contradiction— super smart— and then it plans a targeted solution, shows additional websites that it's doing research on. So this is really in-depth. Okay, it's done processing. Let's see what it came up with. First of all, It came up with an executive summary like you might see in an actual business. So that's super smart. Evaluates the opportunity, specifies the target audience, goes over the problems that they're experiencing, and it gives some ideas. An AI-powered inspector, a web-based tool that leverages pre-trained computer vision models would be an ideal solution for this problem. Wow, that is a super detailed and good idea. Comes up with our pricing, any bonuses. Then it gives opportunity to intelligent data harmonization for shop floor visibility. Talks about the pain point, target audience, and then the idea. This is a web-based tool to automate extract-transform-load process for small manufacturers would be a high-value product. Give you some ideas about how to do a minimum viable product. Super smart. Start with an Excel file or CSV they can load. By the way, of course, Gemini would be a great product to actually write the code to do what it's asking here. Build a backend with Python. You don't have to know how to program, you just have to know and tell the AI what you want it to do, and it will write the code automatically. And it also gives sources, the detailed results. As you can see, it gives fantastic detailed ideas and results here that are very unique and well thought out. It also gives you a conclusion, and it gives you sources of where it looked and did its research to find what it analyzed as the best way going forward and creating a product for these manufacturers. Very, very good results there. Let's go ahead and take a look at Claude. Take that same prompt. I like how Claude goes through and tells you, similar to how Gemini does, it tells you exactly what it's going to be doing, where it's going to be searching. Now, interestingly enough, Claude's response is much faster than Gemini's. Gemini took probably the longest, but it did probably the most in-depth research. But let's take a look at their results here. It's still going to continue to do research and give us results, but let's start. An AI readiness assessment tool. Now that is similar to some of the other results we got, like over on Groq, Scorecard SaaS. It's a little bit different idea there. However, I like the way this is formatted compared to even Gemini. You'll notice Gemini used the number thing as, and it didn't necessarily correspond. This is a little bit better organized, similar to the ChatGPT response where it gave you, you know, told you the opportunity at the top and then went through each of the things that we've suggested they research and give us feedback on. Data prep opportunity 2 is data prep automation for AI. Excellent. Predictive maintenance alerts. That is a very unique idea. One we hadn't seen. I like that. An AI implementation budget calculator. That's actually a good idea. And manufacturing AI failure database. So the other thing I do like about this is in the title they actually put what they think is, is the tool, even though they explain what the tool is below. It's still just is better organized. You're going to find that Claude really gives very good, organized, detailed information back. I'm really impressed with Claude, so you should definitely check it out. And this is once again the free version, so this is available to everybody. Finally, we'll turn to Microsoft Copilot. Copilot is actually using ChatGPT-5 now. You'll see, uses ChatGPT to think deeply, so we should get a similar result from— since the it is using ChatGPT, but we'll find out very quickly. This AI chat from Microsoft, which is Copilot, actually comes with a Microsoft 365 subscription license. Okay, let's take a look. This is actually— it's being trained on different data even though it uses the ChatGPT 5.0 API. Let's just compare those two. Data quality and accessibility for AI models. Okay, that's different. So you can see they come up with different results. Monitoring AI models post-deployment. Yeah, those are different results. So anyway, you can see, and I do like the way this is, this is also responding similar to Claude in that it's telling us the opportunity number and then giving the details within each opportunity. Anyway, there you have it. The same prompt has been put into the different versions of the most popular versions of AI chats that exist out there, and you can see that they give different results. So what can we conclude from this experiment? First of all, you're going to get different results depending on the AI chatbot tool that you decide to use. In this case, since we're trying to come up with an idea for a new product, it's interesting how they pretty much gave a different idea for every chatbot, which is very interesting. So in this case, I would actually use all of the chatbots for ideas when I'm creating a new product. It's not that one is better than another. They just give different feedback and different information based on its training. And knowing this should help you realize that there isn't a single model that will handle everything exactly the way you'd like it to or give you the ideal results every single time. As the different AI models continue to compete, you'll see each one of them develop strengths in different areas. And I don't believe that there's going to be a clear winner anytime soon. The winner here is us as consumers, because each of these models will continue to develop and grow as they compete against the other models, and we just get better results every time. I'm really impressed with the results from ChatGPT. I also thought Copilot's results were very good. Claude is a strong contender, and I really like everything that it's creating right now. And you just can't count out Google with Gemini. It clearly did the, the most in-depth research, and the results came back in a very business-like format. So what would be my next steps? My next steps would be to take each of the results of each chatbot and put all the ideas in a single prompt that I would then ask an AI chatbot to analyze and evaluate which idea they think has the most promise. But now it's your turn. Take a prompt that you use on a regular basis in your own business and put it through each of the popular chatbots you see here and see if you can distinguish one as a clear winner. And let me know what you think below.
