Unlock the Power of AI: Transform Your Accounting Firm with Igor Pogany 🚀

Right.

So my special guest I'm about to bring on very, very soon is somebody who I've been following now for, I think it must be 18 months. When I first got into ChatGPT, when I first discovered it, right at the early stage of last year, I was watching YouTube videos to learn how to do better prompts, how to use it, and I. I came across a number of people doing YouTube channels and doing YouTube, but there was one that was far better than anybody else. I followed his YouTube videos. I was watching them. He then released a course on prompt engineering. It's one of the first courses I bought. He has tens of thousands of people watching his videos. I'll just pull up his channel very quickly. It's the AI advantage, and I'm very lucky to have. I followed him for a year and a half now. And then it turned out earlier on this year that Igor moved out to Lisbon. We became good friends and we're actually meeting up on Friday at my. My place here. So with all of that said, let me bring on Igor Pagani. If I pronounce that right. Is that right, Igor? Is it Pagani?

That's right. I don't know. That's really up to you. It depends on the country you come from. It's from my grandfather's Hungarian side, but Pagani is totally fine. Great to be here, Mark. Thanks for the invite.

It's fantastic to have you here and you're looking great, and I know we've got loads of stuff that we can talk about. I have lots of questions for. For you, partly some of the questions, selfish questions, because I've been out the loop for three weeks, being on vacation, and this world is moving so, so fast that I just kind of feel I'm a bit behind, behind the curve with. With some of the latest things coming out. So I may ask some questions later to find out what I should know about what's happened in the last three weeks. But let's start off, Igor, with the first question just. Just by way of background kind of, how did you get into this space? Because you got this huge YouTube channel, this huge following. You have now, your membership, which I'm part of the AI advantage. But how did you get into AI and ChatGPT and prompt engineering, all this great stuff.

I think it's best answered by kind of my journey before that. And I'll try to keep it concise. I mean, I could talk about it for 30 minutes. There's many interesting details and stories within that, but generally speaking, I was always just an early adopter of tech. Always just a curious kid. When I was small and I got my very first phone, I was that weird kid that went into the menu and had to check out every single menu setting. I opened up the camera and tried pictures on every megapixel setting, the different audio settings, everything. That was just always me. So, you know, growing up, I turned that into my career. I built a video production business first, but next to that, that was not enough. At a certain point, I kind of reached the ceiling of all the technicalities there. I learned about audio production, video production, live streaming, course production, directing, lighting, all of these topics and. But a certain point, you maxed out. So I started venturing into other topics. I started learning about carpentry. I taught myself how to code in pyth. And I was just always curious, always trying to learn. So naturally, when I was in one of my discord communities that were Super Technical, when ChatGPT came out, that was the topic of the day. And naturally I just had to try it out and I gave it a shot. I was aware of things like Dall e before that, but ChatGPT was really the first light bulb moment for me. And mind you, this is literally on the day of release. Okay? So I start using it as I usually do. I kind of obsess over it. I spent the weekend locked at home trying different things. And it was so obvious to me that there's so many use cases here, like, wow, I'm creating all of these scripts, proposals for clients. All of these business activities can not be replaced. But definitely enhanced with the very first version of ChatGPT GPT 3.5 as we know it today, in December 2022. So as I learned these things naturally, I was also running a YouTube channel next to my business. It was more like experimenting. It was my fourth YouTube channel at that point. So I kind of started like a fifth one, AI Advantage. I was like, this is incredible. Let me just turn around and teach the Internet what I've learned here so far, because this seems really powerful to me. And I was really excited when the videos all of a sudden didn't get 40 views overnight, but 80 views overnight. I was like, wow, there's some real interest here. These people want to hear more about it. So I created another one and another one, and all of a sudden, like, January 2023 comes around. And then the world caught on and the world kind of figured out, like, wait a minute, this is really powerful. People should be talking about this. This is really revolutionary. And then the Hype wave came around. Right. So all of a sudden, tens of views turned into tens of thousands of views quite quickly. But generally speaking, I was just a curious person that, you know, I think from speaking from experience, I'm quite good at learning and adapting. It's just my life story just set me up that way. And yeah, I've done that. Every single time something new comes out, I make it my job to be the first one on top of it to figure it out, and then I turn around and teach it to people. And at this point, I managed to scale a little organization around this entire concept of just scanning the Internet for what's new, being the first person to learn it and to reteach it. And along with that comes many other great side effects that YouTube and running online communities provides to you. But that's essentially the backstory. That's how I began. And, you know, also a bit of a workaholic, so, you know, putting in more time than other people always helps. So, yeah, that's the story.

Awesome. Yeah, I thought I was an early adopter, but not as early as you are. Yeah, that was impressive. And, and I'm going to talk about prompt engineering later on as one of my questions for you because that's such an important skill. And it was, it was thanks to you that I, that really took my, my learning to another level because I realized early on when I, when I came across Chat GPT, which I think was the end of, I was a bit slower. I think it was February 23rd, I think I came across it and this light bulb went on. I thought, wow, this is, this is powerful stuff. This can, this can make a huge difference. But we have to figure out how to use it. We have to learn. And that's when I came across this term prompt engineering, which fascinated me. So we'll get into that in a little while. What I do want to ask next. And by the way, some people said there was a bit of echo, which I think I'm going to put my headphones on in a second if I can find them, just to make sure. Although Neil does say it sounds great now, I did change something, but if you see me disappear, it's just because I'm grabbing my headphones. Anyway.

A question I would like to know is, let's start with ChatGPT, because that's where you originally focused. I'd like to look at some talk about some other large language models later. But let's start with ChatGPT simply because that's the one that I Use and most of my audience use ChatGPT. Thanks. Teresa says it's fixed now.

And so what I'd like to know, Igor, is how do you use it in your business? What are the things that you are using ChatGPT for that's having the biggest impact in terms of either saving you time or getting you better results? And then hopefully some of those things that you use it for in your business. The accountants, the bookkeepers listening will think, ah, I could do something, something a bit like that as well.

Okay, it's a great question. I would actually take one step back and maybe abstract from the question one level and really maybe present a little bit of a framework to you and the audience how to think about these things. Because there's really two buckets of kind of use cases. One of them are the one offs, the things that you just kind of want to try and get done, the brainstorming sessions, the hey, here's a screenshot or here's some images I have and let me perform some type of task on top of it, right? These are the one off tasks. And that's a very different approach to something that is going to be recurring, something that you might be some activities you might be performing on a daily basis. I'll give you an example from my business, for example. We use it a lot to manage sponsorships and sponsorship replies, basically email management. So I'm not saying we fully automate the process away and every email gets an automatic response, but we certainly use it in our pipeline. So whenever emails come in, we feed them to a little router that splits it up into a few different buckets and then we run prompts on top of those and then we have drafts that we can work with rather than staring at a blank page. Sponsorships are such a great example with this because it is usually the same thing over and over again, right? We're not reinventing the wheel. They usually want a sponsored segment or a dedicated video. We have different criteria for each one of them. Pricing is pretty consistent, we updated from time to time. But it's pretty templatable, right? And if something is templatable and repetitive, you can kind of build something for it in the second bucket that does it really well. And this is where prompt engineering really shines. I think a lot of people get it wrong and think like, hey, if I'm just, you know, like brainstorming a few ideas or if I'm just creating some sort of draft or I just want a creative partner to bounce some ideas off, I Don't need this fancy prompt engineering. And, and they're right, they're right for the little one off use cases. It's more about raising the awareness of what these tools can do. What, what? Like, like a lot of people like to talk badly about prompt packs these days, but prompt packs, what they really do is they, they show you what's possible. It's about opening your eyes to it. It's not about copy pasting everything you have in there. It's not about, you know, like, I still think there's a lot of value in these libraries of many prompts. You do prompt a week in your newsletter. I do a prompt a week in my newsletter. As we go, we kind of discover these little new use cases that people might not be aware of. And then raising your awareness of what it can do by looking at prompts is fantastic. Following some of the updates, maybe not all of them. I think it's a bit overkill for most people. You know, you can get overwhelmed easily with all the AI news. But just following what OpenAI's making and doing is valuable. Being able to kind of roughly describe what code interpreter does inside of ChatGPT and being aware of the fact that you can upload images is valuable because if you're going to be interacting with it on an everyday basis and you have some base awareness of what the use cases are that it can do really well. And you're aware of tooling and maybe some alternative tools, all of a sudden, well, you're like a mechanic that has like a belt full of tools rather than a mechanic that might have repaired cars before but really doesn't have the tools at his disposal or doesn't even know the tools exist. Because all of a sudden you might find yourself screwing in a screw with your nail or with a coin, whereas there's a screwdriver on the table. But all you need to do is educate yourself a little bit to become aware of the fact that there's a screwdriver. Because this screwdriver in this analogy would be a new LLM or a prompt. If you're not even aware that the screwdriver exists, you're going to be using your coin. But that's probably not the most effective way and you could be saving time and doing better. So two buckets, one off use cases and repetitive use cases. And in terms of one off use cases, you can't just go intuitive, you just talk to it. Just talk to it like you would to an assistant, like you would to a business partner to a Human. I always like the framework of talking to it as a very well paid employee because then you have like certain expectations and there's also a certain amount of professionality and maybe you communicate a little more context than you would to unpaid intern or whatever it might be.

But yeah, that's the base approach. And then for the repetitive tasks, that's a whole different deal. That's where it really pays off to go into the prompt engineering and to flesh out all the context, to dial in all the little details. So if you're going to be running that automation or that prompt for the 15th time, 20th time, 200th time, it delivers consistent outputs. That's really where Prompt Engineer shines in, giving more specific but also more consistent outputs. Because if you just do short prompts, it's going to be all over the place. It's going to be like a bit of a random number generator. So those are the two big buckets, but I'm not sure if that actually answered your question, which was what are the use cases that you actually use? But now we established that, which of these two buckets would you like to dive into? Mark, tell me.

Yeah.

Do you tend to find most of the time you are just doing very quick prompts or do you have, for your business, do you have a whole bunch of templates? Because I know you've written in your notion dozens and dozens or hundreds and hundreds of templates, sorry, prompts that people can use. But have you got particular ones that you use over and over again within your business that work really well?

Yeah, I do, but the problem is like, I'll happily share them with you, but the problem is that they're always hyper specific to the business. Right. So if you're, if you're not, if you don't run a paid community and if you don't use Notion and Google Workspace and Circle to host the community, and if you don't have a YouTube channel, which actually me and you might have a lot of things in common in this department, then a lot of these automations and repetitive things are not going to be that relevant. But basically, as I said with the sponsorships, we have a bunch of automations in place that help us reply and just be more efficient with our replies. And you know, once it gets to more serious stage of the negotiation, like we'll hop on the call and we'll write custom emails, but right in the beginning it's super helpful to just get the ball rolling and to spend literally like 1/5 of the time on sponsorship replies. Rather than, you know, customer replying to every single one. So those are definitely some automations that we have in place where we actually really like Claude for that specific, for its writing style. Whenever it comes to writing style, I think it's just generally agreed upon that Claude's Sonnet 3.5 is just way better than anything that ChatGPT has right now. That's a little tip on the side. Beyond that, we have a lot of community related automations. Matter of fact, we went through and we held a community retreat in Portugal a few weeks ago where the entire team came in and we spent in a neighboring town of yours actually in Colares we spent five days and I held a big like team workshop where we went through every single community member and we created a database where we tagged them based on their interests and we created different group chats and focus groups within the community now. But the point of that is it also allowed us to learn more about our members and it allowed us to create specific automations to maintain that database. So for example, we have a brand new onboarding form and the onboarding form, based on the answers, it automatically populates the database which then automatically tags people inside of the community platform. So we managed to automate this process of like, hey, welcome to the community, what are your interests? And based on what they express in that onboarding form automatically they will get tagged and assigned to the different groups within the community. So they get a customized experience right away, which is very interesting. It's sort of something I thought up over the last few months and it's a great use of a combination of automations and ChatGPT because you do need some intelligence there in the middle, right? It's not about hey, if he gives answer A, then adds them to this list, there's a bit of interpretation, there's a little bit of. Some of the answers are very.

They'Re custom, right? A lot of the answers are like open form and they just type up like a paragraph or two. And then we use ChatGPT to kind of interpret that and make decisions which one of the different categories or which one of the tags should be assigned to this person. So that's how we also use it. And all of these prompts that sit within those automations, well, they're very elaborate. They look more like this rather than like a one liner because we want to give examples of hey, a previous member gave us this interest statement and therefore he would be categorized into, let's say the creative department and the LLM department. And this member gave this answer and we want to categorize them into here, into here and that's where they get quite lengthy when you use examples and fact prompting in the end. So those are some practical examples. Beyond that, I love using automations to. I mean I think compared to you and your community, I'm really like a beginner in terms of accounting. I've been, I've been doing it for nine years manually, but it's always been a bit of a struggle. But I just have very basic things in place. Like I have an email address that all my invoices go to and then all the invoices get automatically saved to a folder that my accountant gets. But like there's always this manual step of me actually sorting through it. And yeah, those are three examples of like automations, I guess two of them involve AI. One is just a simple automation that just saves files to a specific Dropbox folder.

Yeah and I love the fact that the way you explain the different types of prompts that the ones that are templates if you like, and the ones that are just one off and yes, I know yours are very long prompts and I think my audience knows that minor long prompts as well. The ones that we send out in our Monday newsletter, the ones that I share with members, they are sometimes they are when they're in a Word document, two sometimes longer because they include lots of things like examples and templates. So for those people that don't know.

Explain to people, Igor, with prompt engineering, why having within your prompts, if you're building ones that you're going to use as templates and use them over and over again, why are examples so important within the prompts? And you mentioned that with the one that you just referred to.

Yeah, so that's one of the best questions ever, Mark, because like truly that is like maybe the most powerful techniques, especially when it comes to implementing it into your business. Because if you really abstract away from these LLMs, aka large language models, meaning that's kind of the category of these AI tools, right? Like GPT4 GPT 3.5 Claude Sonnet 3.5. All of these are LLMs. To answer the question in chat.

Basically the reason you want to give multiple examples at the end of a prompt or you might want to give multiple examples at a prompt. Because the main Property of these LLMs is actually one that us humans have in common with these LLMs. It's they're really good at recognizing patterns and then reproducing patterns. But what that means is these models have been trained on a lot of training data and they recognize the patterns that happen within that. That's why there's a specific tone and voice to ChatGPT, right? And if you switch on over to another model, there's another tone, right? It's like a different human speaking. A lot of people like to bash on this because like it uses specific words a lot. You know, it uses, let's delve into it or like these specific chatgpt esque words. And people, because it went so viral, people kind of developed a little bit of a, you know, AI written radar and also hate in many cases and fairly so it's just, you know, a lot of people like just use it and use the results at face value. But here's the point. There was probably a lot of the usage of the word delve in the training data of ChatGPT, or at least it has been weighed to use that a lot. And that's why it is that way. It recognized that pattern, so now it reproduces it. But you have some input because if you're prompting it, you're giving it extra, you, you are able to give it extra patterns. So bringing everything I said together, if you have a simple prompt like hey, write me an essay about XYZ and at the bottom you include examples of free essays, well, what is it going to do? It's not just going to write an essay, it's going to look at the entire prompt and it's going to look at your free examples and it's going to consider those when outputting an answer. And all of a sudden the base data is not just what OpenAI trained it on, but it's also your free examples. And if something is repeatable, if something is templatable, then it's probably a good idea to give it not one example, not two example, not two examples. Ideally you give it five, because if something happens once, it's a random occurrence. How are you supposed to pick up on a pattern if you see something once? If you see twice, it could be random, right? If it's three times, most probably that's where the pattern really begins. But at five times, there's no real point in doing more than that. Most of the time, like 99% of the cases, if you see something five times and there's like, just as humans, we can pick up on the patterns within that, right? If a text starts with the sentence, let's delve into it five times, you're going to be able to look at that as a human and Be like, yeah. If someone asks you what are the commonalities, you're going to be like, yeah. Each one starts with the sentence, let's delve into it. So ChatGPT can do the same. So whenever it's something repetitive, just go ahead and say, example 1 colon, insert example 1 example 2 colon, insert example. To do that five times, give it five examples with a certain pattern and then you can expect all the responses to follow that pattern, which is obviously super powerful, because then not every, because then every answer is going to follow the pattern rather than just being random. And then if you put that inside of automation, you're going to be getting consistent results.

And there was, I think there was some research a few months back that said that gave some reasons why 5 was actually the optimum number. Did you come across that?

Yes, yes, yes. And also actually, if you look at all the recent model releases like Llama 3.1, etc. They actually adjusted the main models that they look at. Now they look at MMLU5 shot. What that means is the MMLU benchmark is just, you know, general general knowledge, general world knowledge. But five shot means that every time they give it actually five examples in the prompt, so it's not just a raw prompt that includes five examples because this is how you get the best results and this is how they get the highest benchmark scores and that allows them to market it in a better way. So even the biggest players figured this out. It's sort of, it's not just a theory, it's kind of an established scientific fact at this point. Just give the prompts, give examples and you'll get better results.

Yeah. So I think the learning point here for everybody, if you want to create a prompt that you're going to use over and over again as a template, they then put examples in there. An example of that one that I use from a practical point of view is sometimes I have to video train for other people as CPE training and I have to give them then some assessment questions at the end. And so I just dig out some example assessment questions. So it knows that all my assessment questions are written in a certain way. There's four options, A, B, C, D. And I want it to say which one is the wrong one and the reason or the right one, sorry, the reason why it's the right one. And when I give it five examples, it's just perfect. I can give it any script and it will then chunk out, churn out a whole bunch of assessment questions in seconds, which saves huge amounts of time. So examples really, really important.

Let's talk a little bit, Igor, about different large language models, because normally I talk about ChatGPT. You mentioned a little bit, a little while back that you think Claude is better for, because sonnet's really good now 3.5, you use it for more of the written stuff. And I know Emily who sometimes does these live streams. My colleague Emily's a big fan of anthropic Claude as well. So let's give what's your take at the moment on the current different LLMs? Do you use different ones for different purposes or have you just got one or two favorites?

I do. So I would start by this. If you're just using one, I think still ChatGPT is king because of all the tooling and not just within the app. It has the best mobile app, it has the best tooling within the app. It has the GPTs that are not available within Claude and that are actually, I personally think, quite useful. Not extremely useful, but quite useful, especially if you customize them to yourself. So if I were to use one, it would still be chatgpt, but being the little nerd that I am, I'm subscribed to most of them and if I look at the last two weeks of my life, then I split my attention between really free and I'll okay, so I started with ChatGPT. That's really the general purpose one. But when it comes to writing, so when it comes to like text output, whatever, essays, blogs, emails, descriptions, event descriptions, all of that, I go to Claude. 3.5 Sonnet. I really like the tone of it. It sounds more like a human and it sounds very different from ChatGPT. And I would just state that at this point, like most people on planet Earth are aware of the existence of ChatGPT and they might not be using it regularly. Actually some interesting data came on the out on that too. It's actually extremely low the usage every day, which is surprising because you could talk about that. But.

I think people know ChatGPT output when they see it at this point generally. And Claude is just more human. It's, it's, it's also formulated better. So long story short, when it comes to the tone, I just prefer Claude. So do many people. That's why I use it for that and also for code generation whenever I. So, you know, I'm now, I'm not an advanced coder, but I, I always say I know enough to be dangerous, I know enough to get things done, I know enough to, to fix certain things. And also. Yeah, and I, and I have some scripts that I write and use regularly and whenever I generate codes. Cloud 3.5 Sonnet is also just, just straight up undebatable. The best right now. So those are those and then the third one for research. I really like Perplexity, I think when it comes to kind of sifting through the Internet and looking for new knowledge, I think Perplexity is amazing. So, yeah, those would be my top three and really the only ones that I kind of like return to on a regular basis. Then obviously there's kind of like two more buckets, which is one is all the creative tools. That's a whole completely different conversation. Right. So we're not talking about, you know, midjourney or video generators, audio generators, all of that. And we're also not talking about sort of all the no code tools and tooling like automations and chatbot building, etc. Like that's also a third bucket. But when it just comes to LLM choice, these are kind of the three I would recommend right now. Yeah.

Okay. I might have to try out Anthropic then Claude at some point. But I find that, I find the writing, I find the writing is okay with Chat GPT, it's got better. But that could be because as you know, as you know, Igor, I've, I've trained it in my writing style and it seems to work pretty good at the moment. That's not to say it always will because of prompt creep that things keep changing all the time and who knows? But anyway, it's working well at the moment. But Emily's a big fan. I'm gonna, I'm gonna talk to her and perhaps check it out and use it. Yeah, yeah. Well, I don't know if we'll have time to go into some of the other creative things, but if anybody's got questions and wants to learn more about the tools like video creation and image creation, we can go down that route. But I'll stick for the moment with the large language models with the written stuff because I think that's probably most relevant to accounting firms. And.

Yeah, one thing that I wanted to pick your brains on just to get your view, just because I get this as a common question with large language models like ChatGPT is all around.

Data security, which is particularly important to accounting firms. Client conference for den charting. What's your opinion on the security of large language ones like, like Chat GPT? And have you got any recommendations of best practice safeguards and things that we can be, we can do to make sure that we are safe.

I mean it's a, it's a good question. I'll just straight up state that I'm a bit reckless when it comes to like data privacy and this stuff. Like I'm not. It doesn't keep me up at night that I share some of my, I don't know, confidential or like financial details with ChatGPT. I think if you want to be super safe the best rule of thumb is like don't share anything with these online based LLMs that you wouldn't share on social media, right? So if you have an income statement and you would be comfortable sharing that on Twitter or Instagram or YouTube or whatever, then don't share it with ChatGPT. But that is like the extremely cautious side of things, right? Like practically, I don't know, I don't really care, I just kind of go for it. But privacy wise like sure, if you want to be safe you're going to have to go local. It's just so early in that entire local model space that you're giving up a lot of performance. And the best models are, you know the gap is really large between.

Okay so I have an M2 MacBook Pro, right? Like a pretty a great device, not state of the art like by now there's like you know M3 with 128 gigs of RAM. So what I can run comfortably in terms of local models by the way anyone who's not aware there's also there's these closed models by like OpenAI and anthropic and Google and you don't run those locally, you run those in their clouds and you pay a subscription fee or there's a free model where you send your data out to them and then you get it back. Now in the terms they do states that you know they're not going to be using it for research purposes, et cetera. Like when you're on the teams plan for example, if you're on the free version absolutely they're allowed to use it to train other models and stuff. But who really knows? Not trusting big tech companies is probably a good rule of thumb. So who really knows? So if you want to be super safe then you can run local models. Now the problem is if you in my MacBook, which I don't know was like $3,000 maybe a year or two ago, I'm not sure. So like pretty solid machine, all I can run is a 8 billion parameter llama 3.1 model. That's maybe one of the things you missed over the last few weeks. That came up like two weeks ago. It's very, very good. But it's just not comparable to ChatGPT4O or anthropics models because those are probably somewhere in the range of like 400 to 600 billion parameters. And you're comparing it to 8 billion parameter model by the way, for anybody not familiar with this, there's really like more parameters. The smarter it is going to be, the more reasoning capabilities it's going to be to have, the more training data, the more experience it has, so to say.

So you can run them locally, then you can turn off your WI fi, you know, you're not going to be sending your data over but you're also going to be giving up a lot of performance. So I don't know, I guess that's the alternative. Over time these things will are becoming better by the day, by the week and over time we'll have, I mean Heck, the 8 billion model is probably as capable as the version of ChatGPT when we first got it in November 22, similar to it, but by now it's just, you know, GPT4O is just so much better and there's so much more tooling. So I guess it's a decision everybody has to make for themselves. At the end of the day I don't think there was a single incident, something so far worldwide where some confidential data has been shared with some of these companies and then something leaked or something.

I don't know. But also we're very early, right? We're in like year two of this going mainstream and this hitting the mainstream. So we just don't know yet. So I think for me, I keep it in the back of my mind.

Maybe I wouldn't share, I don't know, things like Social Security numbers or whatever with these models. But when it comes to financial details of the counting of my company for this year, like I don't mind, I'll upload it and then it has the context and I can talk to it like an advisor. So everybody has to make that decision for themselves. But yeah, if you want to go super safe, you're going to have to run local models but you're just going to have to accept that that comes with a lot of limitations. And yeah, that is, that is, that, that is, it's a big question.

So that's the security issue. What about what are the other pitfalls that people should be aware of when using large language models like ChatGPT? What are some of the pitfalls Are there some tips and things we can do to get around them? So, for example, there's been a lot talked about in the past of hallucination. Do you see that still being a big issue, or has that largely gone away? And what are the other pitfalls that we need to be aware of?

Yeah, definitely. So hallucination is, as it's stated in the community, it's a feature. It's not a bug. It's. It's where the creativity comes from. It's. It's similar to humans in many ways. You know, like, if you're, like, if you're going to produce 100 new novel ideas, some of them are going to be terrible. And that's just a part of the deal. You're not going to be like, that's by definition because, like, novel ideas, they're going to be new and they're going to be in like, unexplored territory and you're going to be trying to produce something that hasn't been out there before. So just by the very definition, some of that is going to be completely useless and misinformed. And that's just a part of the process. You can't really be creative without producing terrible stuff, at least in the beginning or whatever. And it's the same with these models. If you want them to have interesting and creative outputs, then you're going to have to accept the fact that they also hallucinate. Matter of fact, there is a setting. If you go through the API, which is the programmatic way of accessing it for anybody listening, there is a slider, which is called temperature, and if you set the temperature to zero, you eliminate all sorts of hallucination. But what you also will find that then every answer will be borderline unusable because it's just always going to give you the same thing. It's going to be very uninspired.

So, yeah, it's a property that is just inherent right now. And that's why a lot of people have troubles wrapping their head around it. Because all of a sudden you might. If we go back to the previous analogy of like the somebody repairing a car having a screwdriver, then it's not just a screwdriver, it's a screwdriver that turns into a water pistol once every 10 times you use it and then you're like, the heck is happening here? Why is this a water pistol? Like, I wanted a screwdriver, like a Phillips head. So it's just something that you have to get used to. And it's Something that prompt engineering can help with. Because as we talked about with the five shot prompting, if you include five examples, you're going to restrict the guardrails on the LLM and you're going to tell it, hey, follow this template every time. And it's going to do it. And all of a sudden it's not going to be able to be like, oh, maybe he meant this. And then it just goes ahead. Just remember, like, when it comes to hallucination, the main thing to keep in mind is that the entire model, especially OpenAI's models, have been set up to be a helpful assistant. And that means they're designed to help you and to not ask many questions. They're just designed to just do it. And what that means in practice is that even with a person, if you hire an assistant and kind of he or she walks into your office and you're like, okay, I would like you to draft me five emails to my most valuable customers.

Go.

And that's their first day. And that's the first conversation you ever had with them. Well, guess what? They're not going to know what the five best customers are. They're not going to know what your tone is that you use. They're not going to know the details of the business relationship with those customers. But these models are set up to do it anyway. So they just go ahead and they'll draft up five emails and you'll be like, hey, that's all wrong. You didn't even have to write names of the customers. And like, a human will tell you. Yeah, because you didn't tell me. And I just did my best. The LLM is not going to tell you that. It's just going to do it. And you're going to be like, hey, this thing is hallucinating. Why did it just call my customer John? I don't even have a customer named John. But like, you know, it hallucinates it up, it invents it, it's creative and it tries to help you at all costs. So being aware of that is good. And then you have like, one way to get around that is to simply tell it. Like, ask me clarifying questions. Like, that's a simple one. And then it's going to come back with like, hey, like, okay, what are the five customers? What tone do you have? It's going to give you some of these contextual questions that clarify the context and it's going to give you the. But it's not set up by default in that way. By default, it's Just set up to just do it and it's going to hallucinate if it has to. So yeah, it's a feature, not a bug. You can tell it to ask contextual questions to get around that. And that is probably like one of the biggest pitfalls.

The other one is the tone. It just has a specific tone and you can mark some great work there on how to get around that and how to customize it. That's another one, I suppose. Another one is the, the short term memory it has. Right. If a human had that short of a memory, it would be impossible to operate in a modern society. Even now, these new models, they have like 32,000 tokens.

Of memory. That's not a lot. Our conversation will be approaching that at a certain point and then it just stops remembering what we talked about in the beginning and you just kind of, you might assume that of course it knows what we talked about, but no, it's a certain window. And as the conversation progresses, if anything is outside of that window, it will just have complete amnesia when it comes to that topic. So that's another massive pitfall. You can't just feed it every Harry Potter book that has been ever written and then talk to it about that. It will be like, wait a minute, you can give me like a few chapters, not a book. So. So it's important to be aware of these things.

And this is an issue. If you have a chat becomes a really, really long chat.

It'S going to get to the point where it forgets the earlier stuff. And usually it's the initial part of the chat with some of the most important instructions that you give that is going to forget those things. How would you get around that?

So yeah, there's a few kind of tricks or hacks. The easiest way is to pick a model that has a longer context window that fits your use case. So at this point, 32,000 tokens is pretty good. By the way, anybody who's not familiar with tokens is sort of, it's fractions of a word and you can just Google OpenAI tokenizer where you can copy paste any text into it and it just shows you how many tokens that is. It's quite simple. It's like a large language model specific thing that you just have to be aware of. It's always expressed in tokens. But basically the easiest way is find a model that has longer context windows. So the Google models might not be best in terms of output quality and reasoning and prompt adherence and whatever and code generation and math and Whatever other categories you could measure them across on, but they have a really large context window. And also they have the ability that you can upload video to Gemini 1.5 Pro. They released a new experimental version a few days ago, which is actually a bit tangential, but on the leaderboards it actually ranks one. And nobody can really explain that. It's a bit weird. It seems like they self marketed it. I don't know, let's not get into that. But the point is there's various models with various capabilities and if you're using ChatGPT, there's a third. I think on the free version there's a. I'm not exactly sure, I think it's 4,000 tokens. On the Pro version you have 32,000. I think Claude, if you're using it has 128,000. And if you're using Gemini 1.5 Pro, they actually have 2 millions, I think has that shipped now it's either 1 million or 2 million, but either way that's the easiest way to get around it. So you go to a model with longer context and yeah, these things are developing by the week. Every week we get new models with improved this, improved that. And one of those improvements is context length. Another very MacGyver esque fix.

To this context limit is that if you know you're approaching or you have a feeling that you're approaching the limit, you can just tell it to summarize the conversation so far. And yeah, if you just think about it in a way, if this is the context window, this entire page, and then.

You have text, text, text, more text, more text, and then it starts moving down with the conversation and everything that gets left out. All the text above here, why am I over here? All the text above here will not be included anymore. So the trick would be that, hey, if you're generating text, generating text, generating text. As you're approaching the end of the context window, you tell it to summarize everything and all of a sudden the summary will be here. So as the context. Wait a minute, I need to readjust this. As the summary moves down, the entire summary is this pen, right? So you can go another 30,000 tokens and the summary will still be included in your context window. That's really the trick. Just summarize our entire conversation in 10 bullet points or something like that and it will do it. Problem with that is if you have 30,000 tokens and you summarize that into a few bullet points, which might be a few hundred tokens, obviously it's going to miss some of that context. So you're going to lose that along the way. But that is sort of a little hack on how to get around that.

Yeah. There's no easy answer, I think, Correct me if I'm wrong, but I think a token is roughly equal to 0.7 of a word.

Yeah. Yes, yes, you're right. You're right, you're right.

That's a great statement you're talking about. If it's 32,000 tokens, that's about 25,000 words, give or take. But of course that's quite hard to know exactly what that. It's kind of. You don't know whether you're going to. When you've reached that within a chat. So unless you're going to count them up and count them up or. I just tend to. I tend to keep a chat going for a period of time and then if I'm going to continue, I'll just start a brand new chat with the original prompt.

Yeah. Oh, I have one more. So we talked about Anthropic and they have an interesting thing that is like different from GPTs than anything that ChatGPT has. They call it Projects and it's simply explained it's sort of a container for various chats that has a knowledge base. So you can like, for example, if you have your writing style document, you can upload that to. As a file to the knowledge base, to the project. And then whenever you have different chats, it's always in the context window of those chats. So whenever you're writing something, you can upload the file once and you don't have to like, re. Add it to your prompt every single time. It's just there. And it's enough to say writing in my writing style, as mentioned in Marks writingstyle PDF. And then it does that and then you can kind of manage it effectively because you don't need to create new chats every time. You don't need to create new GPTs, just re. Upload everything, it's just there. And if you know you're going to be writing a lot and you're going to need like two or three documents of context. Projects are a fantastic way to work through that. If you have a project with multiple chats, then yeah, especially for writing, it's probably the best workflow.

I didn't know that. I've learned something that's great. I'll need to check out projects. Okay, that sounds awesome. Okay. We haven't got many questions coming. I've still got more, so we can keep going. I think there's a question from Meryl that I'll come on to in a bit. There was a question from David about what is an LLM, but that was answered. A large language model. I think we've covered that. But yeah, if anyone's got any questions, please ask. We'll get to other people's questions in about five minutes. I would like to ask.

What do you see as being the. Your prediction now? And it's always hard to predict the future, but where do you think over the next three or four months things are going to start going?

I love that question. I think that's always the most exciting conversation. And, you know, as somebody who's obsessively following this space 247 and has surrounded himself with a team of people who do the same thing, I think I might have a educated guess, let's say. But nevertheless, it's still just a guess. Okay, so, big disclaimer. This is like purely speculative, but I have a bit of a theory. I have a bit of a conspiracy theory. So if you don't mind, you know, let's, let's put on our tinfoil hats on for a second and let's see where this leads. But what I think is that OpenAI has been kind of signaling with every single one of their moves over the past years that they want to build AGI. They want to build. They're very vocal about this. Every single interview with Sam Altman, the word comes up and he states that as the big mission, they want to, you know, they add all of these like other words, like they want to build it safely and, you know, for the benefit of humanity and all. But at the end of the day, they want to build AGI. And AGI is not this form of AI that we see today. Today we see assistants, we see co pilots, we see something that is designed to aid us in our works, in our, in our workflows. Today we see something that is akin to. I'm actually reading this interesting book right now. Key person of influence. I don't know if you're familiar with that. It talks about. And it talks about how, for example, when the microphone was, was released, the people who benefited the most were people with something to say or people that had a great voice and now they could sing into the microphone and it can be amplified. Social media is similar. Today's AI assistants are similar. If you're crap at social media marketing, AI is not going to solve that. It's just going to amplify your. Your, you know, below average in aptitude. Yes, exactly. It's just gonna. It's just gonna, like, yeah, there's gonna be more. It's just gonna multiply that. But, like, if you have a. If you're, if you're not good at singing, you're not gonna benefit from a great microphone. Right? So I think that's where we're at today. Now, to circle back to your question. Where's this leading? I think the endpoint, and that's why this technology is so interesting, is something that has a bit more autonomy, something that has a bit more agency over time. I think this vision of AGI really constitutes of more than just this assistant that has.

IQ or reasoning or autonomy. It also includes something that can actually act so often in the space. This. All of these ideas and all of these, like, words that I expressed are captioned in this buzzword of like, an agent. Because an agent is, like, really different from an assistant. An agent actually goes out there. You tell it, hey, you know, run a social media campaign for me. What happens today is like, oh, okay, like, what kind of campaign? Like, let me ask you a few questions. Let me create a marketing plan for you. Let me give you some suggestions. It assists you. But in this, like, agentic version of the future of AI, it's like, okay, done. And then you kind of can open it up and you can see all the steps it took is like, yes, it created the marketing plan, but then it also went into the Internet and did some research that it also went to Canva and created the assets. Then it also logged into your accounts. It may be wired some money over to buy a subscription. It posted, it looked at the analytics, it deleted some posts, it added some new posts, reevaluated, adjusted its data tone, and it's like, yep, the campaign has been run. That is like the agentic future. And I think this is where things are leading. What does this exactly mean in terms of product releases? Well, I think it's going to be incremental. I think last, what was it? November, we had OpenAI's Dev Day, where they announced GPTs. And GPTs are certainly a step in this direction. They're not just like, hey, here's a chat box and figure it out. It's like, no, Build an assistant that does one thing and you can even give it tooling for actions. Give it some context and tooling, and it's the first step in that direction. I think the next step involves a little bit more reasoning, a little bit more context. Window a little bit more ability to not just write code, but also execute code. Like for example, Claude Sonnet does really well, for example, you really love this. If you haven't seen this Sonnet, one of the amazing things that happened there over the past month is that it cannot just write the code for an app for you. So you're like, hey, build me and Samuraiser running on Anthropic API that is branded with the value Pricing Academy in these colors and it will build the entire app for you. But so would chatgpt. Right? But the difference with Anthropic Sonnet is it will also host it for you. So what they have now is it doesn't just write the code and doesn't just visualize what it functions like and gives you the ability to press buttons. It also gives you a link that you can then go ahead and you could share with friends, family or whoever. And like Anthropic Artifacts. Artifacts is the feature, it's called. You can share it with anybody. So all of a sudden it didn't just assist you in writing the code, it actually wrote it and it executed it and it hosted it on the Internet. And sure, this might be a baby step towards this future where these AIs actually perform actions rather than just assisting you in performing actions, but it's a big difference. And I think this is the direction we're going in. And there's another dev day that has been announced a few days ago. I think it's happening on October 1st and last year with the GPTs. And on the developer side they released the assistance API again. It's this assistant idea. My theory is to round my answer out that they're going to be pushing more in this agentic direction, whatever that means. In terms of concrete details, I'm not sure. They might be building out the assistance API. They might be releasing a new version of GPTs. One thing that I'm certain of is that they're going to dethrone Claude 3.5 sonnet as the king in terms of code generation and they're going to have a artifacts like feature, like the thing that I just described, where it just builds up and also hosts them for you. I'm sure they're going to have that too. But it's going to be steps in this direction of getting things done rather than just assisting you. And I think that's where the space is going and that's why I'm so excited and I want to follow this because all of these skills that we talk about here and all of these little use cases and what models to use these things stack. And if you understand how to create a basic prompt only, that opens the door to creating a more advanced prompt into using something like five Shot or many other techniques that can be used. And you need to know that in order to craft custom instructions that really represent yourself and your use case. And you need to know that skill set in order to build an assistance that does a specific thing, because then you're not defining your context, but the system's context. And you need to know how to define an assistance context in order to think about what kind of tools should my assistant have. Right. And maybe building a GPT. And you need to know that skill set to automate things away. So you just leave them in the background and leave them running. And then we have the conversation about different LLMs and stuff. All of these skills, they stack. And I think when we get to this critical point of this tipping point of agents becoming publicly available inside of something like ChatGPT or the API or Fueanthropic or whoever it might be, it's going to be a bit of a space race. Right. So these things happen fast, I think, at that point. Yeah, it's just, it's really good to have these base skills because they stack and they build on top of each other. And if you're going to be starting from scratch, once that point comes around, that's the point where the true mass adoption is going to come. That's at least my theory. I think there's going to be a critical point where business owners are not going to be able to ignore it. I think right now you can still safely ignore it. Maybe you're going to be giving up like 20, 30, maybe 50% in terms of productivity, depending on the use case, depending on your job, but you can still kind of stomach that and just like power through it. And it's still like, we're not in August 7, 2024. We're not at a point where if you're completely blindsided to a generative AI, that you're not able to compete in the marketplace. I don't think that's the case today. I think three years from now, I would bet on an arm or two that it's going to be a different story and you're just not going to be able to compete. And I think catching up will be a whole lot harder because, yeah, these skills stack and it's kind of a ladder. So, yeah, that's why I'm a fan of like Learning this and teaching this, and, yeah, made it kind of my mission to spread the knowledge.

So, Igor, when you talk about the difference between an assistant and an agent, in my simple brain, so correct me if I'm wrong. In my simple brain, an agent would be where you might take ChatGPT and then embed inside it something that's like an advanced version of Zapier. So in other words, so. So you. So ChatGPT will integrate with your other tech stack and make those things then happen. Is that. Is that right?

Yeah, I think that's fair. I think there's a few components to it, Right? So one. So you can think of it, you could conceptualize it as a small company of people, right? So you could tell it like, let's just take the marketing example. Like, hey, run a marketing campaign for me right now. You would be like, you would prompt it to, hey, act as a marketing expert. Here's my context, here's this, that, here's a few examples, and then it will create a marketing plan for you. But as you said, like, in this future version, maybe this is just one of ten assistants that work for a manager, and the manager has also access to your zapier account, and it has access to maybe your credit card and to your login into various platforms, and it can actually make executive decisions for you, and it uses the input from the assistant to make those decisions. So all of a sudden, it'll go ahead. Okay, we want a marketing campaign. Hey, you get me the data analytics from, like, popular, you know, like YouTubers and Instagrammers in our niche. You go ahead and do this research. You go, you know, look for blogs. Then we bring that in. Then our writer is going to come up with some lead copy. Then our website designer is going to come up with a landing page, going to put this all together. You know, maybe we have to pay some subscriptions or something. Like, Mark gave us a $50 budget to, like, complete this project, and then all you get is kind of a result. So, yeah, there's sort of subsystems, but, like, it just gets really complicated to conceptualize if we don't even talk about. If you haven't even wrapped your head around the concept of, like, running a prompt with a few examples, because that's going to be the subcategory of a subcategory, right? You're going to have a manager and an assistant, and the assistant is going to, like, run one prompt. And if you haven't wrapped your head around that, it's going to be hard to Conceptualize like, okay, okay, why is this new AI app like just getting results and it might not show it to you? Right. It's going to be a bit of a black box. Just like with code today. Like these things happen in Claude's 3.5 sonnet and you don't even need to understand coding, but all of a sudden it builds like an app for you. And that's great. But I'm just saying to get the most out of it, I think you will have to understand it and the companies will do their best to adjust it. But just like running a team is a lot of work, I think also running a team of agents is going to need quite a bit of understanding. And I think the understanding is not going to be maybe psychological and psychological like with humans, but it's going to be more technical like with prompting. So yeah, your understanding there is. Right. But like where it will evolve from there we shall see. And this is already very forward looking.

A new world. We're running out of time unfortunately. And I do want two more questions. I've got one for Merrill, but before we get there, a really important question is Igor, if people want to find out more about you and check out AI Advantage, what should they do?

Yeah, sure. So I think the primary Source is the YouTube channel. Like that's where we. I run a weekly news show called AI News you can use. I like to differentiate myself from a lot of the AI content out there. I only focus on the things that you can actually use today. So I don't talk about the drama, I don't talk about the different things that might be coming up in six months. There's a lot of hype and announcements in the AI space. I ignore all of them. I just show the people the stuff that came out this week and how to use it. And we have a testing team that. And then I present the testing results and stuff. So that's the YouTube side of things. Plus I also announced like if there's big announcements that come out that might not be available, I also create separate videos for those. But beyond that we really simplified it. We have, we have, that's the free site. Then we have a newsletter that you can check out under myaidvantage.com newsletter I believe there should be a link. There should be a link somewhere. And if you sign up to the newsletter, there's I think three or four day onboarding sequence with a bunch of valuable resources. One email is my life story and then a bunch of valuable resources that I'm sure people who want to learn more about this will appreciate. And then the whole thing points towards our paid community where it's higher touch and in there we hold around 15 to 20 events every month. We run lectures, we create free practical guides every single week. And we just make sure that whatever comes out, we make sure you're up to date on everything within the community. We introduced one new thing which is super interesting, which you might not have seen yet because we just introduced it a week ago, but it's app rankings. So even if somebody wants to completely disconnect from Generative AI and they don't want to bother, they don't have time for the events, they don't have time for the guides. Like, sure, it's maybe good to know that it's there in case you need it, but like once a month we go ahead and in all major categories like LLMs, like we talked about here, we have a ranking and once a month we update it. So if you look at the ranking now, you're going to See Sonnet and ChatGPT are King Perplexity and a few more are in the A category and it's ranked with explanations and links and guides below. If you want to go deeper. But you can just go and sail the world for two months, come back, check out the ranking, and you're sort of up to speed on what's happening in Genai. And that's what we do with the community. So that's really all we do. And we're all about upskilling people to keep up with all this madness that is generative AI.

Awesome. I'll go and chat it out. We've got 60 seconds left, so I'll have to. A quick, quick one, I guess Igor, but I should have dealt with. Sorry, Meryl, I'm not sure I fully understood it, but I'll read it anyway. Neither Meryl says, oh, you didn't understand either.

No, no, no.

So.

I tell you what, because of time, I don't know, Meryl, if you could repost that question in my Facebook group, I guess you're in my Facebook group. I can forward it onto Igor and get an answer for you.

I can briefly try. So really like, what the answer comes from is really like two places. One, the training data plus the fine tuning and the weights that OpenAI is responsible for. So that's like what the model knows about. And generally speaking, those are pretty similar from model to model because it's just trained on the wider Internet. And two, what you provided and that's where like prompt engineering comes in. Like you have to understand. Like you have to understand how to talk to a human in order to get results from a human within a company. Right. It's the same thing with this. So if you provide it with good context, a combination of that plus what the model knows is going to get you the results. Yeah. And I think that should. That sort of answers it, I hope. Thanks, Mar.

Awesome. Thanks, Igor. That's been amazing. Unfortunately, we're going to have to dash because Igor's got a session for his community which I'm guesting on in a few minutes time. So, Igor, thank you so much for sharing, so much. Great stuff. I've had great fun. There were more questions I could have asked, but perhaps we'll do this again sometimes. And yeah, and for everyone who's here, thank you for coming on and I will see you all on the next live stream. Let me know in the comments what was the most useful thing you learned. I'll play out with some music as normal so I can read your comments and I'll see you all on the next live stream. Thanks, Igor. See you later.

Thanks for having me. Bye bye, everyone.