EP 294: Why the Future of AI Will Be Built by Non-Technical Domain Experts

This is the Everyday AI show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business and everyday life.

A common misconception about prompting and prompt engineering is you have to be a machine learning expert, you have to be an engineer. And that's maybe not necessarily the right take on it. Right? And even when we talk about the future of building AI products, it might not be those machine learning experts per se that are the ones pushing this to the finish line and beyond. It might just be your domain experts, the experts in the field. All right, I've got hot takes on this one, but I've got a guest today and we're going to be talking about why the future of AI will actually be built by non technical domain experts. I'm excited for today's conversation and hey, this is it. Welcome to Everyday AI. What's going on, y'? All? My name's Jordan and I am the host of Everyday AI and this is for you. We are your daily live stream podcast and free daily newsletter helping everyday people learn and leverage generative AI to grow their companies and to grow their career. So if that sounds like you, thank you, thank you for tuning in. If you're on the podcast, as always, check out your show notes. We'll have a lot of more information on today's show as well as a link so you can go read our newsletter. All right, so one of the things we do every day is we recap the AI news. So if you haven't already, please go to your everydayai.com and sign up for that free daily newsletter. All right, so let's get started with what's happening in the world of AI news. So first and foremost we have the Pope Francis is advocating for ethical AI at the G7 summit in Italy. So Pope Francis addressed G7 leaders at their annual gathering in Southern Italy, focusing on the need for stronger guard rails on AI to ensure ethical development and use. The Argentinian Pope emphasized the importance of AI being more with human like values and compassion, mercy, morality and forgiveness to prevent unchecked risks. So Francis called for an international treaty to to regulate AI development, echoing worries about AI safety, potential bioweapons creation and disinformation spreading. All right, next. Well, OpenAI has added a pretty big name to its board of directors. So retired General Paul Nakason, the former head of the NSA was appointed to open AI's board of directors. So Nakasone is to join OpenAI's sake safety and security committee. Also focusing on AI's role in cyber security. OpenAI is obviously aiming to strengthen its cyber security profile through AI to detect and respond to threats quickly. His expertise will likely guide OpenAI and ensuring AI benefits all of humanity. That is their mission. Right? So OpenAI's board now includes Nakason CEO Sam Altman and others notable and other notable figures in the tech industry. Last but not least, Microsoft is shipping its new Copilot Plus PCs next week without a key AI feature. So Microsoft is delaying the release of their new Recall feature after receiving way too much backlash from privacy advocates and security experts. So Recall, which essentially takes non stop screenshots of a user's action on their computer and then allows them to tap into that. Well, it's not going to be available initially to Windows insiders or buyers of Copilot plus PC. So Recall reportedly was developed in secret and not tested publicly with Windows insiders, according to reports. This move comes after Microsoft President Brad Smith testified in front of the U.S. congress Thursday over AI and security concerns. So in that testimony, Brad Smith said that Microsoft is committing to adopting all recommendations made by the Congress's Senate Cyber Safety Review Board, investing in cyber security initiatives, adding more security engineers to the team, and ensuring that security is a top priority for all aspects of the company.

Wow.

Apparently every single piece of AI news today is about security. All right, so there's always more. So make sure to go to your everyday AI.com Sign up for our free daily newsletter where we will be recapping all of that news and more. But today we're here to talk about the future of AI and how it'll be built by non technical domain experts. I can't wait for this one. Please help me. Welcome to the show. There we go. Our guest for today is Jared Zreich, who is the finder founder of Prompt Layer. Jared, thank you for joining the Everyday AI show.

Thank you for having me excited for it.

All right, yeah, this is a good one. I'm excited about this. But before we dive in, Jared, just tell us a little bit about yourself and Prompt Layer.

Yeah, totally. So Prompt Layer, we're a platform for prompt engineering and maybe we could talk about what Promptman Engineering means later. But basically we are helping teams build real AI applications with domain knowledge. And we're small. We're a small company. We're based in New York about like five or six people now. And yeah, it's a, it's a fun time.

Yeah, love to see it. Hey, Prompt Engineering. Can't wait. This is one of my Favorite topics and hey to our livestream audience joining us, Woozy and Brian and someone from Florida and Tara. Thank you. But yeah, please get your questions in for Jared. What do you want to know about the future of AI prompt engineering? It's Friday. Sometimes we get a little wild on Friday, so get your questions in. But let's just talk about that. Jared, what the heck is prompt engineering? I say it is both fake and the most real thing ever. But I mean, what is prompt engineering for those that maybe aren't aware?

Yeah, I'll give you my definition. Tell me, tell me if you agree on this one, Jordan. But basically how I define it is there when you're building an AI system you have a lot of inputs that go in and you have an output. That's all. I define it as tuning what inputs are to get the output. So that includes the prompt, which is probably the biggest part of this. But it also includes what model are you running, what hyper parameters are you sending to the model, what order are you sending it in, all that fun stuff. So basically it's just what do you send to the AI you're engineering those inputs? Input engineering.

Yeah, I love that. And you know Jared, kind of like my, my take on it and I think the easiest way to describe it is like Mid Journey, right? So if you look at the, the early days of Mid Journey and AI image platform, you really had to speak to it almost in code to get anything out of it, right? And now you can speak to it like a human, right? So to me prompt engineering is almost the conversation that the end user has with an AI platform to get the most out of it. And that, that looks very different, right, Depending on what platform. But you know, I'm curious though, let's, let's just get to the bulk or the, the crux of this conversation about this concept of actually non technical domain experts being the key in the future. Jared, what's, what's your take on that and what the heck does that even mean?

Totally. Totally, totally. So I, so I think at the core of this point that, that I guess we're going to talk about here is, is the question of how do you build an AI product that succeeds and how do you build an AI product that differentiates where, where in the world. You know, there's a, there's a common derogatory word people call startups of ChatGPT wrapper and the, the big question is what, how do you build an AI application? How do you build the AI product that is different than a ChatGPT wrapper how do you build a. And my, my take on it at least is that domain knowledge is the way you do it, domain expertise. So I, maybe an example is the best way to illustrate this. If you're building an AI like a legal application, a legal AI app, the, the engineer is building the application. If I'm an engineer, if I'm building a legal AI, I don't know if the contract it's spitting out is correct. I don't know what the correct answer is to a question about a contract is. I don't, I don't understand anything about the law. And the, what this means is you have to have someone who does understand on your team. You do need these, we call them domain experts, you could call them subject matter experts, you call them whatever you want. But in my opinion these are the people who are going to be behind AI applications in the future. All this little like prompt tuning and as you were saying, these mid journey language, you have to speak to the model in that I think is all approaching zero. That's all going away. It's getting easier and easier to build these systems and the one thing that is going to remain is how do you actually impart the task to the AI? How do you tell your AI application, what do you do? Like I think there's a, there's another, there's another tangential concept here which is like, is prompt engineering going away? Totally. And like how are you going to build these applications when it, when it does? And I think one core part that is related here is that at the end of the day, okay, so you're building a travel assistant AI and you say book me a flight from New York to Chicago. There's a lot of different correct answers to that. There's a red eye, there's a three stop, three leg flight. And at the end of the day you always need to define the right task to call it lost function and ML, call it whatever you want. But someone needs to define the task and that's where we go back to the domain domain expert. Someone has to define how you're actually solving the problem. And yeah, that hopefully that makes sense.

No? Yeah, and I love it. And I'm just going to spit out some hot takes here and get your thoughts.

Jared.

So you know, I feel with, with large language models you, you know, especially in 2024 there's been this emphasis on rag, right? You know, kind of bringing in your, your own data and I don't know, I think the future of large language models are going to be small models. I think it's going to be, you know, maybe where now someone's relying on, you know, 90% of a, you know, trillion parameter large language model with 10% of, you know, their domain expert data. I think in the future it's going to be the opposite. I think it's going to be 90% of their own kind of first company data is what I call it. With 10% of the large language model, is that wild to think or are domain experts in the knowledge that they have just too important for that not to be the future? Are you still running in circles trying to figure out how to actually grow your business with AI? Maybe your company has been tinkering with large language models for a year or more, but can't really get traction to find ROI on gen AI. Hey, this is Jordan Wilson, host of this very podcast. Companies like Adobe, Microsoft and Nvidia have partnered with us because they trust our expertise in educating the masses around generative AI to get ahead. And some of the most innovative companies in the country hire us to help with their AI strategy and to train hundreds of their employees on how to use Gen AI. So whether you're looking for ChatGPT training for thousands or just need help building your front end AI strategy, you, you can partner with us too. Just like some of the biggest companies in the world do. Go to your everydayai.com partner to get in contact with our team or you can just click on the partner section of our website. We'll help you stop running in those AI circles and help get your team ahead and build a straight path to ROI on gen.

So I don't, I don't think they're mutually exclusive. I, I guess I think for, well, I'll start with saying first I think this take of everyone's going to be using small models. 50, 50, I could totally see it. I could also see a world where, you know, OpenAI, anthropic, Google, all the, all the big models are just so good that you kind of just use them for most of your things and you tweak, do these last mile tweaks which we're doing today, it's all last mile tweaks, it's prompt engineering, it's fine tuning, it's ragged, I guess how it relates to domain experts and how that'll exist in this world of if we go down the route of everyone using small models, maybe they're using Llama, maybe who knows what they're using. But I don't think it changes anything because if we go back to the definition, prompt engineering, it's also choosing which model to use. And there's at the end of the day, at the end of the day you're getting an output from an AI and someone needs to know if the output's correct or not. And there's a lot of different options you get, especially if, and I think this is the most powerful thing about language models is that you can solve problems that don't have a ground truth solution, for example, conversations. And someone needs to tailor that voice, Someone needs to tailor what it is. And choosing small models, maybe using small models in conjunction with big models. I think it's almost not relevant to the argument of who is going to be building these systems.

I have more on that, but I want to get to this question here from Yogesh, former guest. Thanks, Yogesh, for the question asking, do you see prompt engineering becoming a core skill set that everyone needs, or is it going to be specialized experts who offer their knowledge to others? Love this one. What's, what's the answer there, Jared?

Yeah, yeah, I like this a lot too. So there's a term I read a long time ago, it might have been Stephen Wolfram who said it first called computational thinking. And that's what I think is the real skill set here. And I think that's a skill set. I think that's already almost a core skill set in most knowledge work people do today, which is just kind of can you think algorithmically? Can you think in terms of like, might be, you could call it logic. There's a lot, there's a lot of ways to describe this. But can you, can you reason about how to do something in a computational engine from a computer, from a language model, something like that? And can you build this algorithm? So having said that, yeah, that I think is a core skill set. I think the skill set of prompt engineering is kind of just a mix of communication and communication and this computational thinking. So yeah, you'll probably need it. I think the, in your question, the specialized experts versus prompt engineering, I think that's one and the same. I think the specialized experts are going to be the ones who need to understand how these systems work because they're going to be using a platform. And I see, I see it as simple as in the future. And we want to build prompt layering to this like a platform where you should only, like the only relevant parts for building an AI system should be this specialized expert coming in and just telling the AI what it needs to know. That's It, I mean, all this other stuff we're doing, fine tuning, whatever this is, all can be made easier in the future and can be like, as I said, like approach zero in terms of complexity. The real irreducible part of this is the specialized knowledge and how you impart that to a system.

You know, one thing, one thing I, I talked about with someone recently is someone who was kind of, kind of scared of AI, right? They understood how large language models were useful, but they looked at their own job in their own role and they said, well, this seems like a large language model could do the majority of my role, you know, And I kind of said, well, hey, you have the data, right? It is your decision making, it is your domain expertise that ultimately matters the most. So Jared, like, I'm curious for, because I'm sure there's a lot of people, especially people listening or watching the show today who might feel that way, right? Who might feel, hey, I'm a subject matter expert and you know, if what's, you know, is, is my knowledge going to become commoditized? So for those people who are non technical domain experts, where should they be investing their time in terms of learning new skills and how, how to find ways to still push their companies or departments forward?

Yeah, yeah, no, that's a great question. And kind of like tangentially, who, who are the big beneficiaries of the AI revolution and how do you become one of them? I think these type of technologies and these type of, let's call it like technological revolutions, they rise, they rise the tide of productivity to a new level of certain people. So like, and skills, so like a calculator lets you do math at a higher level, Photoshop lets you do art at a higher level, Cars lets you move at a higher level. I think LMS lets you do conversation and knowledge sharing at a much higher level. And who's gonna be, who's gonna be the maestros of this and who's gonna, who's gonna take advantage of it? And like you're saying, yeah, say, say you are realizing 80% of your job can be automated. It's kind of scary. But that means also someone in that field is going to be not, it's not 80% more productive, but whatever that, whatever that fraction is, I can't think right now, but they're going to be an order of magnitude more productive. And I think the skill set you need is just what I'm like, communication and, and this algorithmic thinking. So use ChatGPT and if you're good at it, start using it more and check out the GPT widgets if you're not technical. I think that's, I really believe like, I think that's a direction we're going to be moving in the future. I don't know if OpenAI's implementation is the right thing right now. It doesn't seem like it really picked up, but I have certain GPT GPTs as they're called for like writing docs and stuff like that and just get a feel for the technology is probably the best thing to do.

Yeah, you know, speaking of, you know where don't non technical domain experts. Right. Which I think is so many people, you know where or you know, how can companies kind of leverage these people, you know, for companies that are investing in their own, you know, whether their own fine tuned large language models, whether it's, you know, you know, software companies and they're investing in their own products and services around AI. How can these non technical domain experts help set those products apart?

Totally, totally. That's a great question. So I think we, let's, I think we can look at, let's look at like the AI applications that exist today and maybe talk through those. And that, that, that should elaborate here. So there's like a group of AI applications that are, that I use to code, for example. So like cursor is a great one obviously copilot. All these like AI tools that just make like software engineering an order of magnitude better. These I think are a confusing example to look at because the people who made them are the experts because they're made by engineers for engineers. So, so let's ignore all those for now. And the other subset of AI applications are kind of like the Harveys of the world, the Hebia for example for financial model for financial companies or character AI for like Personas. All of these applications rely on some like bringing AI to a specific vertical. And I think if you're building an AI product today, especially one not in software, in software it's a little bit easier because everybody on the team knows what it's supposed to do. But if you're building an AI application in a different vertical, there's two ways to do it. And one way is to rely on engineers and kind of like do the best you can. But the way to really build a good product and stand out is to actually have the domain experts in charge of the project. So I can give some more examples. So there's this company, Parent Lab, very cool company, we've worked with them for a while. They are a coaching app, like a parent parental coaching app. And they basically the person who's in charge of the AI product, or, I don't know, in charge, but in charge of prompt engineering, the AI product is an educator. She actually never touched code in her life. She was a teacher for 16 years, did real estate. But she understands like communication and she understands how to talk to people. And it's incredibly important for them to get. Yeah, this is it. It's incredibly important for them to get the voice right of the product and for the product to make an actual connection with these parents. So that's a, that's, that's a big example of like, if you, if you want to build a really good product, and this is a very unique product, you can't. You need to have someone, you need to bring in that domain knowledge as your competitive edge. And there's so many other examples. There's Gorgeous is another great example. It's the number one help desk for Shopify, for Shopify stores. And they want to make customer service really good and they're trying to automate a lot of it and save a lot of money for merchants. And they're building this AI product and the people. So at that company, we work with both the ML team and the prompt engineers, and the prompt engineers are largely customer support specialists who are then coming in and saying, hey, this responded in the wrong way. This promised too much. We don't want it to do this much. And yeah, they're really building like the cutting edge customer support chatbot. And the way they're doing this is by leveraging these people who actually know what the responses should be as opposed to people who know how to use Git version control, you know.

You know, Jared, I'm, I'm curious, you know, getting, getting more to this concept of, you know, just prompt engineering and those people who, because I don't know ever. Right. I'm not like, I haven't been around for forever, but I've been working, you know, full time since I was a teenager. So, you know, more than 20 years. And I can't remember any other time, at least in my kind of working history that there was such a, it seemed like such a rush to adapt to a new technology.

Right.

It wasn't really like this with the Internet. It wasn't really like this with cloud, it wasn't like this with mobile. But with specifically generative AI and large language models, I don't think there's ever been such a disparity between kind of the. Haves and the have nots in terms of the skill set. But for those people who are still non technical, right. When we talked about prompt engineering, I think it's just this scary term. So let's just break it down. How can a non technical person be good at prompt engineering? Like what makes a good prompt engineer?

Yeah, yeah, it's, it is, it is very interesting. It is creating a new world of like, there's like a gap of technology where like innovation has happened so fast and now there's, there's a lot to do to catch up to it. But yeah, how to catch up, how to be a good prompt engineer? A lot of it, yeah, a lot of it boils down to this communication skill of are you good at articulating ideas in words? Are you good at articulate, like understanding? LLM's large language models are a tool for language, for processing language and outputting language, and that's a big skill here. And honestly, familiarity with how it works, I think it's really like, it's closer to the skill of hackers and makers and that sort of thing as opposed to just raw engineers. A lot of times, like people who are good at prompt engineering, you'll see it split in an engineering team. Half the people are good at it, half aren't. And it is a completely different skill set. And to get good at it, I'm not sure the best way to get good at it, Honestly, I think, I think find out if you're like decent at it and then just start doing it a lot. But to go from 00 to 1, honestly, I'm not sure, I'm not sure. Just maybe, maybe using ChatGPT more and more.

Yeah, yeah, I agree. It's one of those things. It's like you can read and watch YouTube videos about riding a bike and, you know, talk to people who are experts at riding a bike. But until you do it right, and until you start to fall a little bit and see why you fall where you fell, et cetera. But yeah, I just think it's, it's, it's being curious and it's having conversations and it's, it's iterative work, right? Because I think people, when they think of just the word prompt, they think of like a long, long prompt, right? And they say, oh, it has to be very formal and it has to be structured and in JSON or YAML, et cetera. And it's like, no, it's have a conversation with a model, right? You know, especially when we see, you know, people like you Know, Sam Altman saying, you know, hey, the future of large language models are going to be much more conversational. They're going to be much better at picking up on kind of like in like human intuition. With that in mind, you know, how, how would you suggest people to suggest to people to prepare for, you know, large language models to come? I mean, is it, is it best to just, you know, go in and just keep prompting until you fail? I mean, what's the best way to improve on those skills and learn to ride that bike?

Yeah, totally. And I think there's two ways to answer this question. There's the, there's are you, how are you doing the prompting? Are you building a application? Are you building an AI application and you're doing prompting to build this application, or are you just doing it for yourself? So maybe I'll start with doing it for yourself. That's a simpler question. Kind of just, it's not up to you. You know, the models that are come out, are going to come out. But in general, all the, I guess when activity first came out, there was a lot of buzz of, oh, if you're nicer to the model, you get a better answer because stack overflow questions that are nicer got better answers or something like that. That's probably true, but it's even, I doubt it's even true today. And a lot of these tips and tricks not even worth learning too heavily because they're going to go away. Exactly. Like you said earlier, conversational is the way to go. Models are really good at fixing problems. So then I guess like the second way to answer it is how do you prepare for new models if you're building an AI application? And how do you not index too heavily on what exists today because you want to build these things in a model agnostic way? 4.0 came out, people had to change their stuff. A lot of our customers actually had to change it back from 4.0 because it wasn't working for their stuff. So the way I think we actually call this concept prompt routers, which is build things in a modular way. Instead of relying on one prompt to do everything and relying on this AGI like prompt that is just an autonomous agent running in the background. Build a flowchart, build a state machine where each prompt is doing something specific that you can test modularly. Then it's super easy. Swap out the model, swap out the new model, see if it's faster, see if it's cheaper, see if it works better and you can build these unit tests and I think like rigor. Rigor is the answer for the second group is, in my opinion.

Yeah. A great question here, following up from earlier from Cecilia, but I think it's important to talk about. So she's asking and saying, can you give a more specific example of computational thinking and what is the best way to develop it? Yeah, that's, this, that's a great question here, but yeah, what's. What is that, Jared? What is a good example of that?

Yes, totally. So a good example of computational thinking is. Let's go back to that example I gave earlier of booking a flight between New York and Chicago. When you hear that question, the way to think about it computationally is to say, what are the exact steps I need to solve? And this is actually funny enough, this is how people say you should be prompting with chain of thought. So it's kind of chain of thought for humans. So the first step is saying like, what are my requirements? What does this person need? What are the options? Just how do you. In other words, computational thinking is how do you break down a problem into computational steps, into discrete steps to do. And the best way to develop this skill is kind of logic math coding. Coding is honestly probably the best way to learn how to think computationally. It's not necessary, but the whole concept of coding is writing, doing a problem in discrete compute steps. Not to use the word in the definition.

Yeah, so that's, that's a great. Yeah, like even just thinking chain of thought. Right. Like I think a lot of times when it comes to getting the most out of a model, it is just thinking of those important steps. Right. And sometimes, you know, you have to give an outline or work backwards from the solution that you think you need and kind of provide those stepping stones to help a model get there and have a conversation. So Jared, we've, we've covered a lot today, right? Like we've talked, you know, about kind of the future of AI models and how non technical subject matter experts are the key. We've talked about how the tech revolution rises, the tide of productivity. We've covered a lot. But maybe what is your one important, one most important takeaway as we wrap up for people to best understand kind of how the future of AI is going to be built by non technical domain experts.

Yeah, put me on the spot with one important takeaway. I think, I guess the one thing I think people should understand is AI applicant. The. Yeah, the differentiation differentiators of good AI applications are bringing experts into the loop who actually know if the outputs are correct. Or not. And obviously I say that's what we do at prompt layer. I think there's alternate takes to it. So if you disagree with me, plenty of products that do disagree with me. And yeah, that's what I say. Bring experts into the loop because you need to build an AI application system with people who know if the outputs are correct or not.

Yeah, that's huge. Hey, still always room for smart humans in the loop. So yeah, if you're non technical domain expert like so many of us, don't worry. Still still needs you.

Right?

All right, so we covered a lot today and we're going to be recapping it as always in our daily newsletter. Jared, thank you so much for joining the Everyday AI Show. We really appreciate your time and insights.

Thank you for having me, Jordan. This is fun. All right.

And hey, as a reminder, we covered a lot if this was helpful. If you're listening on the podcast, please drop us a review and subscribe to the pod. Also, if you're listening on LinkedIn. Thanks y'. All. Tag someone that needs to hear this. Repost this if this was valuable. But most importantly, go to your everydayai.com we're going to be recapping today's conversation as well as going over what you need to know to keep up and stay ahead in the world of AI. Thank you for tuning in and we hope to see you back for more Everyday AI than.

Thanks, y'. All.

And that's a wrap for today's edition of Everyday AI.

Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going for a little more AI magic.

Visit your everydayai.com and sign up to.

Our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.