Do ChatGPT Detectors Work? Let's Find Out!

Hey, what's up? Matt Wolfe here. I've recently been hearing a lot about these GPT chat detectors. You know, these tools where you could copy and paste an article or some sort of text blurb into this tool and it will tell you if it was written by ChatGPT. I know a lot of people are really worried about Google and how they're going to detect this stuff and whether or not they can use AI content inside of their blog to post and their content marketing. Is Google going to derank it? That sort of thing? So, so I got really curious and I recently came across this Reddit thread here which has actually been deleted and the person who posted it is no longer a user on Reddit, but they claimed that originality AI is a huge scam. Now, even though this was deleted and they took it down, I don't know if they saw the error in their ways or if they just flat out got removed from Reddit, I don't know. But it got me thinking, how well do these detectors work Now? I have some content out there on the web that I know was helped with AI, and then I have some content out there on the web that I know I personally wrote myself. So I thought I would put some of these AI detectors to the test. I've never used one of these tools, so I'm going to test them out for the first time in this video and you're going to see my process here and I don't know what the results are going to be myself. So let's go ahead and dive into this. So I went to Future Tools and I typed in detector up here. And These are the three most popular AI detectors that I've come across. We've got originality AI, we've got detect GPT and then we've got the GPT2 output detector, which is a free one, supposedly detects GPT2 output, but also still works fairly well for GPT3 output. So I'm going to put these three tools to the test and see what we come up with here. So the first one that I want to test out is the GPT2 output detector. This is a completely free one that you can find over at OpenAI OpenAI detector HF space. The easier way to get to it would be to go to the Future Tools website and just, you know, search GPT2 output detector and you'll come across it. But I'm going to test this with some content that I know was written by AI, as well as content that I know was not Written by AI. So, funnily enough, most of the descriptions on the Future Tools website was written by AI. So this description for detect GPT, no, this was actually written by AI. Let's post it into the GPT2 output detector and it's going to predict real quick and let's see what it says. So it says 96.67 real, 3.33% fake. This originality.AI also written here. Let's go ahead and copy this and paste this one in here and let's see what it says. So this one says 97.56% real, 2.44% fake. So this was written with GPT3. And since this is a GPT2 detector, it didn't seem to notice that it was fake. All right, so now I'm going to jump over to my mattwolf.com blog. This is all content, purely written by me, no AI involved on this website. I'm going to go to this one called Everything is Changing from October of 2022. I know there was abso0ai involved, so let me just go ahead and grab a couple paragraphs at random here and copy them over to our GPT2 output detector, paste them in. Let's see what it thinks. 99.97 real, 0.03% fake. So it knows that that one's pretty real as well. All right, so GPT2 output detector probably can't be too relied on since it is designed for GPT2 content, not necessarily GPT3.3. Let's go ahead and try one more thing. Let's go into Chat GPT and prompt something real quick and see what it gives us when we plug it into the GPT2 output detector. Write a story about a wolf that learns to use a computer and then makes videos about how to use artificial intelligence for productivity and making money. Let's go ahead and have it write a prompt that we know was 100% AI generated. All right, let's go ahead and copy this story that it just created and put it in the GPT2 output detector. This will be our final test with this one. Let's see what it says. All right, 0.2% real, 99.98% fake. So the stuff that's coming right out of chat GPT, the GPT2 output detector still seems to be able to detect it pretty well. All right, so now let's test the Detect GPT tool here. Let's click on Visit Detect GPT. This one is a Chrome extension, so I'm Going to go ahead and install this extension here, add it to Chrome and now we've got it up here. Let's use this same originality AI text because I know for sure that this is AI generated. Copy this. Go ahead and click this thing up here. Let's go ahead and paste it in here. Click Check Content according to this is 2.44% AI, 97.56% human. So we know that's not true. Let me pick another tool as well. I know this autodraw. This description here is also using GPT. Let's go ahead and paste this content in and check it. And this one is showing 57.78% AI, 42.22% human. It says we are 57.78% confident that this is AI generated. So a little more than 50 50. All right, so let's go ahead. Once again, let's go ahead and grab this same selection from my blog post, which I know is 100% human generated. Pull it up into our content checker here. Check the content. And this is saying 99.93 human, 0% AI. So it's getting my blog post right. All right. And then one last check. Let's copy this here that we knew we pulled straight in from Chat GPT. Paste this story in here, Check the content and it knows it's AI. So this one seems to work pretty well as well. It seems like the shorter text, it has a little bit harder time detecting it. These one paragraphs here from my Future Tools site. It's kind of 5050 on whether or not it's detecting it real well. The last one I want to play around with is the one that actually charges for you to use it. I'm hoping that this one is the best one that's out there because they charge $0.01 per 100 words per scan. Now, I don't have an account, so I'm going to go ahead and sign up for one right now and we'll run the same test through this one. So I'm just going to quickly jump through the signup process here. I'll go ahead and purchase some credits. I don't have any credits right now, so let's go ahead and grab some. So let's go ahead and run our first scan here. My scan. I am going to grab my autodraw text here, come over to Originality AI, paste it in, Go ahead and detect AI. Sure, why not? Let's do a plagiarism check as well. It'll cost me two credits. So Go ahead and scan. Now, according to this, it's 93% original and 7% AI and no plagiarism detected. So interesting result because I know for a fact that that was generated with AI and it's saying that it was original. All right, let's go ahead and do a new check here. Let me grab the originality AI. This was definitely 100% written by a AI. Let's go ahead and run a check on this one. Both detect AI and plagiarism scan this guy. This one says 100% original, 0% AI. There we go again. I know originality AI was 100% written by AI. I know for a fact. And I didn't run it through any other tools. I didn't do any spinning, I didn't do any extra stuff. This was just purely 100% generated through AI. I promise you, I wouldn't have used the wording. It is available for a low price of $0.01 per 100. All right, so now I'm going to go ahead and grab this text here from my blog post, which I know for sure, sure is not AI. And let's go ahead and run a new scan. Let's paste this guy in here, scan it now for both AI and plagiarism and see what we get. So this one says 98% original, 2% AI. So I got a better score on the one that I know for sure was 100% AI than the one that I know was 100% not AI. Interesting. So let me. Let me grab a different chunk from the same article. Copy this. I'm going to try one more here. This is the one that's actually charging you money to use. So I want to be fairly thorough with it. 99% original, 1% AI. So I mean, you know, it sees it as original, but the fact that it thinks it's 1% AI is still interesting to me. So let's go ahead and check this story about the wolf who makes money online and see what score it gives me. If this thing has wants any credibility, it better not show it as original. So this one, it detected 100% AI. So it does seem to be. Seems to be working for the most part, although it didn't seem to do very well with some of these shorter text blurbs here. Let me see how many credits have I used? So I bought $20 worth of credits here and I've used whatever that adds up to be. 10, 12, 14, 16 credits out of my 2000. So here's what I'm going To do one last time, let me open up GPT3 here and let's have this OpenAI. Write me a GPT3 thing and let's see what it gives me. Explain to me how GPT3 works. All right, so we've got a few paragraphs here written by GPT3. Let's go ahead and copy this, pull it into originality AI, paste it and scan it now. 94% original, 6% AI and even found a little bit of plagiarism. So that's kind of interesting. It's saying that it plagiarized this blog post here. Let's go ahead and take the same prompt. Let's plug it back into this GPT Detector plugin. Let's paste that in here and check the content here. 99% AI, 2% human. Those are the comparisons I played around with Detect GPT GPT 2 Output Detector Originality AI. After playing around with these three, my feelings are that GPT 2 Output Detector is probably the least accurate just because it's kind of looking for the older GPT2 technology, but it still seems to be able to detect stuff that came out of chat GPT originality AI seems to be the number two. It kind of thought some of my blog content had a little hint of AI to it. It also thought some of my AI content had a hint of no AI to it, which I found kind of interesting. And then the one that I would pick as the winner of these three that seems to be the best at detecting is this Detect GPT plugin, which is a free Chrome plugin that you can use. So I installed this one. If I'm going to be searching for GPT, I'm going to be using this detect GPT. Now I might use originality because I got $19 worth of credits now because I couldn't buy less than that. So I'll probably use this from time to time. But the one that seems to be the most accurate and find the best for my experimentation has been this Chrome extension which was free. There you go. Those are my thoughts. Want to find more tools like this? Make sure you check out futuretools IO adding new tools to it every single Right now we've got 350 total tools in here. As an added bonus, if you scroll down to the bottom and you put your email in here, you'll be able to grab the free AI Income database, which is all sorts of cool ways to make money using AI. And you'll also be added to our weekly newsletter where every Friday I send you five of my favorite tools that came out that week, three interesting news articles, three cool YouTube videos, and one new way to make money with AI. You're going to want to be on that mailing list. It's growing really fast. Join almost 2,000 people who have already joined that weekly newsletter, and I hope you enjoyed this video. Go do some AI stuff. Have fun out there, kids. See you in the next one.