Is Prompting The New Programming? with Filip Kozera and Robert Chandler, Wordware AI

Great. So in 2011, Mark Andreessen wrote an article, a blog post, which every aspect of life— and fast forward, what, 13 years? It has everything that we take in when it comes to information. Every aspect of our digital lives now is done or has aspects of software in it, including the traditional industries. We've been in that phase for a while, and it feels like we are now getting into that next phase. This whole AI wave, we've done a couple episodes already on that, on the GPU and the hardware side. This episode, we are actually going to be talking a bit more on that AI software side.

So just as you said, the Marc Andreessen article that was posted in what, 20— I don't know, 10 years ago, 2011. Yeah, yeah. In 2019, somebody who's been quite consequential in this space— yes, software is eating the world, but AI is going to eat software. Right. So when Shikhar talks about this kind of next phase, we're looking at so many things that happened in the last, I don't know, 2 years that has involved AI. People have never heard of AI and its practical applications are now hearing about ChatGPT. People are using ChatGPT for marketing efforts. Gemini has come out. There's so many things going on in the AI space that may be replacing some really functions. Now, Shikhar was mentioning software development is one of those spaces that's like a black hole for folks who don't really understand what's going on in the industry, right? Luckily for Shikhar and I, we have a ton of friends who are SDEs who are very familiar with this space. And so learning about how AI is transforming that space has been super duper interesting, right? So as AI begins to eat software enhancing and accelerating certain development processes. There's tools that facilitate this integration becoming very crucial. Enter Wordware AI, right? A platform that's designed to streamline and simplify the creation of AI-driven applications, which is becoming quite popular these days. The removing kind of the complexities of traditional programming, allowing certain domain experts to streamline and simplify the creation of these applications. Today, we're super lucky on Things Have Changed Podcast to have Filip Kozera and Robert Chandler, CEO and CTO of Wordware AI, respectively. Welcome to Things Have Changed Podcast, gentlemen. Welcome. Absolutely. So we'll get right into it. Our first question is really around this AI development space, what do you think's the hardest thing about developing AI-driven applications? What are some of those problems that you're hearing from some of the folks who try to use Wordware?

Yeah, so I think we are inherently working with something which is non-deterministic, which brings a lot of difficulty. We've had to rethink how programming language, will change once we take English as the core of it. And this is what lies in the heart of our product is rethinking how can we work with something which is inherently non-deterministic and about hallucinations and there are things which are good about the non-deterministic nature, like hallucinations actually in my vocabulary, it's almost equal to creativity. Working with these things which change and shift and we no longer can do a very strict if statement. That's definitely very difficult for people to grasp. And sometimes changing one word is enough to change the whole output of a large language model.

Prior to LLMs, if you wanted to automate some process, you had to gather a dataset of millions of items. You had to get people to annotate them. So to give you the correct output, given some set of kind of noisy inputs that would take months in and of itself. You then had to train a specific model for one specific task, which would require a team of quite highly skilled machine learning engineers. And then finally you get this model and then you'd have to keep it up to date and keep it in production. And so it only meant the most high-value, most frequently done tasks could be automated by big teams. With LLMs, that's changed. Suddenly you've got these foundation models that are capable of actually reasoning but it is incredibly easy compared to how it used to be.

Nice. Uh, GPT, Mistral, whatever, right? And it feels like anytime I try a same prompt, I might get different results, right? Is that, uh, so that's just probably 95% of the use case of the general public right now.

For—

are we seeing something similar with the devs who are trying to build these applications and they see such a wide range of variability, or it's probably hard to get a certain fixed solution that they need for their application or fixed workflow that your solution was needed, the Wordware solution?

Yeah, I think there are ways to combat the high levels of variability. So a lot of the chatbots that you're using on the internet, ChatGPT, Claude, Perplexity, they've all got a fairly high temperature setting. So they'll give you output rather than a completely deterministic output.

Which robot?

The OpenAI deterministic setting thing does work, but you have to get a signature from their, like, one output and pass that into the next output. And yeah, it works, but inherent to the randomness of these models that in order to get the same outputs, you have to run it on the same type of GPU in the same data center with the exact same inputs, which is where this signature thing comes in.

And in our tool, you just have a slider being like temperature. And we're like, okay, but it's set to zero. And then we had to dig a little deeper to exactly figure out what would make it deterministic.

Got it. So there is some level of, um, variations still, even after you narrow down the window. Jumping straight into how Wordware came up with the solution, what were you seeing on the ground? Because I just, by reading some of the Notion documents that y'all are building, y'all have been in the AI space for a while, even pre-2015s, 2016s, where this was still novel in a way compared to today where Everyone's on AI right now. What did you see or experience in your work that led to this product, the solution?

And it's exposed in a collaborative web-hosted IDE. And that's a lot of words, but effectively what it means is you get a place that looks like Notion, where actually instead of creating documents, you're creating programs. But these programs are sequences of prompts is Actually, what's underlying everything you see that's called an agent or everything that you see that's called an AI application, they're all just chains of the cool other prompts that do some self-reflection, whatever it might be, but it's natural language. And so what WordRite does to help kind of this non-determinism problem, solve this randomness problem, is give people the ability to iterate faster and faster. If you're changing prompts in the codebase and you have to spin up some new instance and trigger that and then show the outputs to someone who's maybe more knowledgeable in the field that you're trying to automate, which is often the PM, but it could be like a lawyer or a marketer or a salesperson for those kind of industries. Putting them in the driving seat and getting them to be able to iterate, that speeds up the development process infinitely. And so they're able to get a feeling of like, when is the model outputting things in a space that I don't like? When's it sort of, you know, going off track, being too noisy, and they understand which words to change in their prompt to make that better. And whilst they're still going to be completely random— not completely random, still going to be largely random. It's a language model. It's inherently random. You're able to guide it to a much better set of outputs on a broader set of inputs. And so it's just having that human in the loop in the development process and being able to do hundreds, thousands of iterations even like thousands of iterations a day that leads to better quality LLM applications.

You mentioned marketeers, salespeople, and whatnot. Are they— the audience, I'm, I'm sure covers a broad stroke of software developers as well. But when you talk about these marketeers and these salespeople, do you think that the goal is someday they could fully utilize Wordware by prompt engineering, or do you always think there's gonna be an element there of ML expertise that they need to go learn?

Yeah, so maybe I'll take that one. Essentially, I think we have a paradigm shift and what we are at the core of the problem that we are solving is we believe that AI will bring more engineers, not less. And it just, the definition of the word engineer will change a little bit to somebody who is extremely accurate in using English. And knows how to communicate with these monsters we call LLMs, because the way that they actually work, we don't get it. So often I imagine this kind of multi-tentacle monster that's holding two puppets, which are looking like human, and this monster is playing with these puppets. So these people need to understand these monsters. And by the way, the puppets come from reinforcement learning from human feedback. This monster learned how to imitate humans and how to talk to us. And in order to actually understand the whole monster, I think there will be engineers who are using English as kind of their main programming language in tools like Wordware. And we're already seeing the paradigm shift. We're already seeing people who are called AI engineers. These people will work with domain experts in order to create actually fully functioning AI agents. So I think there's going to be people who understand both sales and collaborating with sales specialists. I can change a couple prompts and we'll have AI engineer who maybe helps with the exact styling of the prompt or how does retrieval get into the whole chains that we are building in programs like Wordware. So overall, I think enterprises are starting prompting teams. And yeah, so one of the, one of our clients, actually a big enterprise, 7,000 people, they are starting a 100 to 150 people team. That will specialize in LLM interaction. The way that it's going to pan out in the future, I think it's hard to say right now. I think these people need to be somewhat technical, but they don't need to be coding in Python. They just need to be very good at communicating their concepts clearly. And that's difficult because our minds are scattered. And I think even on a podcast like this, with a question, answering it immediately in English in a precise manner is difficult. and it will always be communicating between LLMs and humans, always be difficult.

That's so fascinating, Filip, that already you're seeing this. I know one of the core visions that stands out from just your website is prompting is the new programming. So it's fascinating that you're already seeing that. Coming into this call, I thought, yeah, I can see it. I put these prompts and I get these results. But you are saying that now if you just take out software engineer, you just take out the programming aspect from it, they're still solving problems and you can probably solve problems through the prompts. Is that the idea?

Yeah, I would say I'll go a little bit further and currently everyone understands prompting as just a singular piece of text above a generation. And yeah, I would kind of expand it to say actual future and the equivalent of what you spoken at the beginning of like software is eating the world right now. It's going to be agents are eating the world. And maybe I'll just do a quick explanation of what agents are just for the rest of the context of this, because they are at the heart of a kind of AI revolution. So far we've understood AI as something that retrieves knowledge either from the weights of its network that's been trained or from some document we're using where we're putting that document into vector database, which is a more complex database that can approximately know the meaning of each document without looking into the whole, reading all of it and retrieving knowledge. So those are the use cases we're getting right now. Chatbots that can chat with you and give you good answers based on some body of text or the weights of the network. What's coming next is AI agents. So AI agents are actually capable of completing more complex tasks. So I'm going to give you an example around essay writing, even though the whole of AI agents can actually do a lot more. They can actually be like copilots for every part of our world and our lives. So if we tell an LLM right now, write an essay, it's like me sitting you in front of a computer, putting a gun to your face and saying, hey, you now need to write an essay about the Roman Empire. You cannot Google, you cannot use any tools, you cannot plan it before, you cannot talk to your friends, and you cannot reflect, which means you cannot use delete, backspace. And I'm putting this gun to your head and I'm saying, right now, and the cadence of each word should be the same. Go. And I think that essay would be pretty shit. And it's incredible how well LLMs do at that. But the most important part right now, and what we are introducing as the core of Wordware, is an ability to create these agents. And what are AI agents? I think they consist of basically 4 skills. So reflection, in this case, it's like using backspace and actually being able to go back if you wrote something shitly. Planning. So before you write an essay, you actually are allowed to sit down and make sure that the whole thing works, that you can do research in particular things. The third one is tool use. So you can use Google and you can use maybe Grammarly for your, for how grammatically bad you are writing. And the last one is multi-agent collaboration. And multi-agent collaboration is the kind of more complex one, and it essentially means you can talk to people who are better at research and better at Googling and better at grammar and style and those more specialized agents. So I think this is where we see this kind of agental AI coming into play. And even though I mentioned essays, right now it's becoming workflows and it's becoming helping with people's work and making all of these decisions, maybe not fully autonomously yet, but at least being able to plan them out in a specific manner.

Filip, I wanna say one thing. So I read that to make your answers a bit better and a bit more human, you gotta add things like, please help me, thank you for this, and stuff like that. So does putting a gun to the LLM's head, does it make better? Does it actually make it better by adding things like that? Please help me, and you're such a nice guy, or such a nice girl or whatever.

Yeah, I think these, there's so many of these, these techniques. I think those with time will go away. Having the ability to just talk to it in a more natural voice, I think models will improve at that. So you don't have to say these kinds of things, but at the core of it, planning the whole flow of it, being like, hey, start with planning. You should think in this way, have a thought, think about an action, and then conduct the action. Those kind of logical, almost like logic-based steps on how to think will always persist. I'm actually looking right now at a 12— oh no, it's actually way more— prompting techniques. So you've got zero-shot prompting, few-shot prompting, chain-of-thought prompting, self-consistency, prompt chaining, tree-of-thought, active prompt, program-aided language models, multimodal CLT, graph prompting. It's right now, it's a lot. And that's why we are saying that currently it's, it's a full-time job to know all of this. What you've mentioned are these kind of little things that help. There's a whole world that you don't know about that, that is just— yeah, I can even send it to you later. There's 1, 2, 3, 4, 12, 17 different categories of how to prompt, and in which each one of them there's so much more.

It's so interesting because As you were explaining that, Filip, a lot of questions were going through my head on, on what to ask next and how to basically frame what Wordware does and where AI is going in general. And initially I thought, okay, prompting is the new programming. Programming has been an abstraction, isn't it? It's just more of making things simpler to be able to speak to a computer and to give it commands, right? So we've created these language, these languages on top of that, these programming languages to make it easier and easier. If you think about it, Python is much easier than the first programming languages. I met somebody the other day that was telling me about, God, I can't remember what the name was, but they would be coming out of these massive machines and punching holes into them or something like that. I forgot what language it was.

Punch card programming. It's literally like punch card programming. It's the, yeah.

That's insane. Like people actually used to do that shit on the binary. So it's an abstraction. Every level we get better and we get better and we get better. And then when I first saw this headline on Wordware AI, I thought to myself, okay, that's another level again of abstraction of saying, look, We're getting closer and closer to where you just speak to a computer and it gives you what you want. Now you're talking to me about this kind of agent, this agental AI concept that's coming up. I had never thought about this prior to this call, by the way. Is that somewhere that Wordware is trying to get people to where you're able to give multiple tasks to this one prompt and be able to bring them all together to get the output that we want? I think that's a really key difference here. And I think Shikhar, when we make the blog for this, I wanna really focus on agential AI. We haven't covered that concept before, but it's just so novel for us. But yeah, I guess I'll pause there. How does that tie into kind of Wordware?

That, that is, is the biggest thing about Wordware. It's like giving people the ability to make these more agentic loops, uh, in their prompting. You can kind of think when you use ChatGPT, you can do the same thing Filip was talking about with the essay. You can give it time to plan, you can paste in extracts from Wikipedia, but you're there in the loop kind of going back and forth with it. Whereas if you kind of distill that into a program, you're able to just click a button, put in the Roman Empire and get out a great essay a few minutes later. And we kind of, when we were explaining to non-technical people, we kind of think about when you're using ChatGPT and you're doing that back and forth, you're a wizard and you're casting spells and you're kind of like, going back and forth, waving your wand, seeing the results, putting in some new inputs, muttering some new incantations. And then when you use Wordware, you turn into a potion maker. And so you're able to distill that spell into just like a reusable thing, a potion that you can share with your friends or you can drink or you can pour on your data, whatever, however you want to take the analogy forwards. And it basically turns everyone into as powerful a prompt engineer as you are just by drinking the potion.

And to add to this, I think what you've mentioned is assuming generalized agents. It's like, how do you build these agents that can think, et cetera, et cetera? We are not there yet. Agents like AutoGPT are gimmicks and they are cool demos. Where right now we're seeing real business value is these handcrafted agents. You are actually as a human and a domain expert guiding it. Firstly, plan. You're giving it time to plan and then say, critique this plan. And you might use a different model to critique this plan, so to change it. Then you say, hey, you have these kind of tools because of the plan. Do research, research each one of them. And then we do a loop. So do you see how these kind of classical programming concepts like looping and conditional statements and calling a function being like, this is a function for planning. It consists of all of these things that somebody in Wordware community has created and it works super well. And I would not be able to come up with the right prompts for all of this. And you're doing this kind of potion, as Robert said, but you're not saying to an agent being like, yeah, you can plan, reflect, and use tools, go for it. But you have to guide it in a very specific manner. And that's why we're super excited about GPT-5. And by the way, I think you never want to get involved yourself with AI companies that are afraid of GPT-5 or 6 or 7. You want to get yourself involved with the people that are super excited about it. About it. And that's why we're so early still. GPT-4 is amazing, but for what we are using it for, which is not token generation, but a reasoning engine behind something which is trying to take human creativity and, and human productivity and put it together, that's going to be the next frontier. So for now, it's very much humans behind the recipe of the potion, but at some stage it might be other agents.

So Filip and Robert, on that point, you mentioned a lot from the Wordware community having some ideas of ways to improve their applications and their building of software. My question would be related to that, where are we going to see— I do know there is a promise of AI touching every aspect of the, of what we do in our lives, right? Like how software ate the world, right? But for now, coming back to like our first point, are you seeing more of that AI impact on the software industry first? And is it down to just the fact that it's already like the structured data, that it's already been trained on all this data, that it's easier to see the impact right away on the software side? And that's how solutions like Wordware are being built.

I actually think it comes down to software engineers being domain experts on software. And so the reason we're seeing loads of software-based applications is because software engineers know how to build software and then validate if the output of the AI that built the software is any good. The reason we're not seeing loads and loads of applications in legal tech, in medicine, in law, in marketing, in sales. I mean, we are seeing companies spinning up to do this, but it's not as thriving a community is because most lawyers and PMs and marketers don't have the ability to import LangChain into their Python environment and write a chain of prompts and connect it to some kind of internal data store. I think living in the Bay Area, you think everyone can code. If you leave the Bay Area, you realize actually it's not a skill that people learn as kids just from their parents, because most people's parents can't code. And yeah, I think that's what, that's what literally what Wordware is there to unlock. It's, you know what, there are so many creative people and we're seeing them in the community. They're not technical, but they are empowered by using Wordware to build not just toys. So there's a lot of low-code tools that allow you to build like simple flows, very like you can chain one prompt to another prompt and then use DALL·E or something. You can't build anything actually useful there. Once you get the ability to have the same kind of concepts of programming, but in an interface that's familiar to like non-technical people, the creativity explosion is incredible. And so we are super excited to be this like GitHub but for prompting. Like we're still super early days building out all these community features, but on our Discord there's people sharing prompts, like cloning each other's prompts, being like, oh, what if we change this line? And oh, it worked great. It now, like then someone else takes that and improves it. And it's magical to see. And these are not like non-technical people taking their world knowledge, domain knowledge, taking just experiments that they're excited about and building cool things.

I think there's absolutely an appetite for that, right? Because right now all these creative people using AI is some form of interaction with a consumer-facing LLM. And right now your workflow is, look, I gotta write this paper or I gotta do this deliverable. I need some advice maybe from ChatGPT. They're gonna input something in there. They're gonna take one of their employees, I don't know, 6, 7 page paper and put it through an LLM to see what's going on with it. It's fractured because if I wanna do something else, I gotta go to another tool and go use that tool with the same input that I'm gonna get from GPT. And it's just interesting that unlocking that creativity, when you talk about unlocking that creativity, there is, we know there's groups of people who are willing to learn how to code, who are willing to learn how to do these like small things. And there's always gonna be a limit, right? For example, like going to college, I also took a bunch of coding classes and it's not because I wanted to become a software engineer, to be honest with you, just a lot of people that I respected encouraged me to take data structures. Just because they told me, look, the world of tomorrow, you're gonna need to understand data structures and tools like this, I think will make that much easier for people like me who are going into the space who occasionally— I have no software development engineer in my job description, but occasionally I have to write some code. It's the world that we live in today. Tools, I think like Wordware make that a little bit easier to take it to the next level. It's not just coding now. It's not if-else and loops. It's literally dealing with LLMs. So dealing with them in a more workflow-focused area, which I think that's absolutely fascinating. Are you finding that— sorry, just before you react.

Yeah, go ahead. Yeah. Sorry.

Oh, I was just gonna say, are you finding there's a ton of use cases for people who are trying to get interested and learn more about your product on how to interact with LLMs? Sorry, go ahead.

Yeah, and I think one of the problems that you haven't mentioned is that people even if they have done a little bit of software engineering and their brain does work in this kind of more analytical way and you can put a structure on it, can't deal with things like deployment. They can't show other people that their, the results of their work, which is just really sucks. If you take a person who can do a little bit of Python, but then you tell them, hey, spin it up on the website with a frontend and put a bunch of LLMs together. They're just going to panic. And this is where Wordware comes in, is we are building the first community of people where they can take their prompting and take their ideas, create these workflows or kind of little blocks of LLM interactions and share them with the community. So with one click of a button, if you, first of all, if you are a CEO or an early stage startup, or you're trying to build a product with one click of a button, you get an API. Which you can plug into the code. And this is for some people, this is their whole backend. So they spin up a frontend, they put the API there. And because you can run code on Wordware, you can actually do everything on top of it. But there is another way where you just click share and that gives you a hosted application in a similar way to replicate.com, which allows you to host your machine learning models and show them off to the world. Here you get to show off the pro— your, your kind of Word apps or prompt cascades with tools, et cetera, to the rest of the world. And also very importantly, you, you allow the community members to fork that and to duplicate that flow and to reuse it inside of their application. So I think you very correctly kind of identified that there are people who are analytical and you've even probably wrote more code than we expect our users. To be able to do. But yeah, we're giving it to all the domain experts. So it's not only going to be code gen right now. A farmer can write a program which will try to optimize how he should move the crop based on the weather and a bunch of factors that I have no clue about and neither any software engineer has any clue about. So that's why we want to, for these AI aficionados who are early on and they're intelligent, analytical, and know their domain, Give them the right tool to be able to, to their knowledge and instill it and give it to LLM in the right manner.

It's a little bit like Excel in that Excel is basically the most used programming language in the world. It's probably got the most developers building with it and it's not a programming language, but it is a programming language. And the reason it's so popular is there's such a long tail of applications that, yeah, there's no way startups or companies are gonna build the thing for you, for everyone. You end up with a regression to the mean on what companies are viable. And it's where is there enough value and like on a per usage basis and enough users, multiply that together and you get a company that you could actually start and build. But there's a limited supply of software engineers. There's a limited supply of entrepreneurs for now. Once we get these AI engineers and software engineers, that might change. But yeah, you need, you basically need to put the tools in the hands of the people that know their problems, know what kind of thing they would build if they had the tool to build it, and then actually give them the tool and see what they build and empower them.

That's super interesting because I just, so when you say Excel, Jed's eyes light up because when you, yeah, he's that guy. He's that guy. Everything in his life from like grocery bills to stock modeling and performance of an asset is all on Excel. The first thing. And it's funny because the first time I met him, he was doing all these calculations of like utility spend and stuff like that and how we are trending over 12 months and I'm like, dude, that's, it's up by a dollar. What are you doing? Trend analysis.

Dude, get a girlfriend.

It's, yeah, we used to, we used to, we used to live together and I'm like, we live together with Robert as well. He's downstairs.

You just do the same thing. It's, it's important. Before you, you ask the question, Shikhar, I just wanna point out, I just, it's, you mentioned Excel as a coding language, right? Py— there are about 8.2 million people around the world who know how to code in Python, right? If you were to guess, just tell me. Yeah, it's probably too low. There's probably some people. But if you were to guess, how many people use Excel in the world? Just throw out a number. I just want to see what the perspective is here.

300 million, maybe 150 million. 200 million, something like that.

There are 750 million people in the world who use Excel.

That's our target market. That's our target market.

And those people are analytical. They're analytical, right? Inherently, you have to use Excel to do a lot of the business worlds. I think in 10 years, people will think about Excel as, oh, that's like some tool that a lot of the older people use. I disagree. There's a lot of people who are still going to continue to use Excel because they can't get to that next level of abstraction. I think Wordware can be that tool for folks who want to utilize LLMs, for example. Anyway, sorry, Shikhar, I completely cut you off.

Yeah, no, no, I was actually going there because I wanted to convey that I use Excel a certain way, and Jed being, you know, a financial dude for the hedge funds that he's worked, he's used Excel in a different way, and he loves using it that way. So I was just coming down to just Wordware as well and how you've mentioned that you want to be like a Notion-style collaborative environment. So one thing that, you know, all our THC research is on Notion, and we got those templates from the community, and it has helped us, you know, turn those notes into a product, which is this podcast, and where we actually connect founders to VCs and capital and stuff. So it's so interesting how you're going at it the same way where you're creating this community. The community shares a certain prompt and someone else can build a product based on that automation or workflow.

Can I, yeah, can I just ask, you're mentioning Notion. Yes. All our docs, by the way, are on Notion. We love Notion. This was really interesting to us just to see, cuz we have hundreds of docs on Notion about each company that we've interviewed and how that prompt. So for us, it's like an obvious choice. If you wanted to get involved with LLMs, I think Wordware would be one of those tools just cuz it's very similar to Notion. But can I ask like, why Notion? Is that something that you guys have fallen in love with and like it as a productivity tool? Or do you think it's just made things a lot easier? Cuz for us, that's the use case.

Case.

For us, the use case is Notion has made the way we write docs easier. It's simpler for me than going on Microsoft— I don't even know what it's called anymore— Word, right? So Notion has been that tool for us. But yeah, why Notion, I guess?

Curve, because it is, it's still a learning curve to use Wordware. Like it's a new paradigm. You're having to firstly learn how to do prompting and then how to use our tool. I think hopefully the how to use our tool part is easy and you just basically need to learn how to do prompting.

Prompting.

And that's why we see people that have used like LangChain, they can move to Wordware and be like, oh my God, this just makes things way lower friction and way better. And then people who are like, have not done any prompting or agent building before go, hmm, still confusing, but I get the slash command and I get the like kind of how to use the tool. I just need to work out how do I prompt. The other kind of why Notion and why not something else. So there's, you've probably seen a few low-code tools that are flowchart-based. And actually the very first iteration of Wordware was built at a hackathon, was flowchart-based. The problem you reach with flowcharts is whilst they look very simple when you look at them at a high-level blockiness, when it comes to actually like building something complex where you have these loops that are going back round, you have branching, your simple-looking workflow that was like a few blocks leading one to the next turns into this horrible mess of lines going all over the place. And it's actually way easier and way— like the structure of software is that it's like documents and you have functions that call functions, which gives you effectively a third dimension rather than this kind of 2D grid of a flowchart-based thing. And so that was really— it came through seeing the deficiencies in the flowchart model that we were like, what if it looks like something else? What if it looked more like programming? And maybe the way to make that accessible is to make it look like Notion. It was very much came out almost like a design experiment. And then it was like, oh wait, it works.

This is beautiful.

Like, I can't tell you how much I enjoy prompting in Wordware compared to writing prompts in the codebase.

Yeah.

And I think a good example here is imagine a triple loop. So a loop inside of a loop inside of a loop. And imagine, try to imagine that as a block-based thing. It's very hard. And sometimes you do need to do these things where you have a database of owners and each owner has a cat and each cat likes to eat something. And as a company or as a person, you might want to run some LLM analysis on, for example, each one of these cats, trying to figure out if the cat is fat and what is it doing wrong. And then I'm just coming up with this right now. Imagine trying this loop inside of a loop inside of a loop. In programming, it's a beautiful concept. You can see how it's doing that. In 2D, you just run out of dimensions. And Robert has mentioned that like functions give you a third dimension. Actually, there's no limit and bound on that. You can, I don't think if, if many listeners to this podcast intuitively understand multidimensional spaces. I'm going with yes, that many understand it and it's actually giving you infinite dimensions that you can play with.

At the same time, I, we assume yes as well. We're going to give that credit to our listeners. Thank you. At the same time, thinking through kind of your target audience, right? Like folks who are creative, who want to get into the technical stuff and learn just a little bit enough to build something beautiful. I'm thinking about it at an enterprise level where you mentioned, right? Like PMs really use Notion a lot. Shikhar and I are PMs, product program, whatever. We interact a ton with the software engineers at our company. They're the ones creating some really cool stuff as well and all that. And guess what? When we get into those technical conversations and they show me what they're working on, I don't know what the fuck's going on. Okay. I go in there and I'm like, what are you trying to tell me? And they try to make me docs. We are at Amazon. We love docs. So we write a ton of docs. It's not on Notion, but we write a ton of docs. So I try to force them to write a doc. They don't like to do that shit. I'm supposed to write the doc on all of the technical stuff. When I initially looked, I watched your walkthrough, Robert, on creating just a chatbot or the first agent, I guess. That just to me was like, okay. If I can work with them like this at an enterprise level where, okay, let's say we have some kind of LLM integration and I need to communicate with domain expertise as a software engineer, that actually makes it a lot easier for me to even communicate to them in English. This is exactly, I think, what I'm doing. This is the structure that I need it to be. And I don't know if y'all are thinking about templates just like similar to how Notion is doing. I think that'd be a huge unlock just for people who don't want to think about the structure, don't have to think about certain things that that software developers have to think about. I think that's a huge unlock. So you're probably also thinking, you mentioned customers that might be on the corporate level, really big companies rather than just startups. Is that also a target market for y'all? Like these folks that need to be working on those workflows?

So you've mentioned kind of two different ways. So one is the community sharing templates where it's the same as Notion, but actually in corporate setting, the dynamic is a little different. So because you can write some code in Wordware, what these engineers are doing, they're providing you with these blocks and you don't need to understand how these blocks work, but they will tell you, hey, we figured out that for retrieval of this type of legal, we need to choose these hyperparameters and these vector databases. And you don't need to understand any of it, but they're going to come to you and tell you, hey, when legal documents use this block, we called it retrieval for legal documents and we've made it work perfectly. And then there will be different retrieval for internal documents just because how much of a magic spell retrieval augmented generation is still. And so we see kind of engineers writing these blocks and you as a consultant or as a salesperson being able to prompt and play around with it and see exactly what the retrieval has outputted. And create these, these well-working Word apps. And because you can see everything, it's at the core of it, we are a what-is-what-you-get platform, which means that you can debug it extremely quickly. So those kind of, those, this is what we are seeing from the corporate world is that they are collaborating and it's not like the same as Notion template because the Notion template would be like a fully working thing, but it's more these two people working together. And yeah, it's a lot about consultants. As well. So they go to the client and they listen to their problem and they are a little bit better at prompting because I think consultants' jobs are going to change from creating decks to creating proof of concepts right now. And I've got my own theory about why they are freaking out right now and why is it moving so quickly. I can dive a little bit deeper into that, but they essentially booked a lot of revenue. And in order to create good return on investment for all of these companies, they need to be doing something else than the PowerPoint deck.

Yeah, on that, Filip, I think our next question kind of leads directly into that, right? Y'all are the experts, Filip and Robert, and I see that both of you like to do this a lot where you want to know, you are hypothesizing, you have predictions that you envision a year, 12 months, 24 months ahead into the future. So I'd like to get your thoughts as we jump more into the industry side of things. How do you see, do you have some top 3 predictions for the next 12 to 24 months within the space that you all are experts in?

Shikhar is asking, as we have in our notes, all of your LinkedIn posts on your predictions.

I need to pull up that Notion page that I wrote about 6 months ago. Was it future predictions?

Yeah, some predictions.

Some predictions.

Yeah, you called it some predictions.

I think the next 20 to 24 months is going to be in San Francisco. Everyone is talking about agents. And I think the way to compete in the future is to move to San Francisco and go to a bunch of events. And the stuff that, yeah, the stuff that we're talking about, is going to come to the ever, everyone, everyday customer in 12 to 24 months. I think on hypotheses, I think AI agents are going to be huge. I'm actually not sure. And this is, I don't know, I'm just, I think everyone believes it's such a clear and obvious thing, but I'm actually not sure if on the corporate level, just doing vector databases and retrieving knowledge from the stuff that's been written and connecting it to a chatbot, but that would be a good return on investment. So that's another part. I don't know if that, if Gleam is going to be successful. I think the jury is still out on that. And next on the kind of more business stuff, and Robert will dive into the more technical things. I think the consulting companies that are not embracing AI and are not creating POCs and are not leaning in like Accenture, for example, leaned in, they booked a $600 million of GenAI-only consulting in the Q1 of 2024. They are on track to do $3 billion in consulting on just GenAI. So essentially everyone's freaking out. Nobody knows about what this GenAI thing— we definitely need to do something with it. We don't know what to do with it. I think we're going to start, we're going to start to see a lot more kind of corporate use cases outside of these kind of just ChatGPT, et cetera, et cetera. And I think that the heart of it will be probably software like a world where the next 50 million of engineers, and you will see these kids who are training not in Python, and we'll see a lot more courses being like how to create an AI agent using just English. And this is on the business side and on kind of the societal side. Next 12 to 24 months is going to be transformative. And we can jump into kind of what that means in the longer term. But on the technical side of things, I'll let Robert take off, take over.

Yeah.

Yeah.

I, I, I like that. I found the document that, that you guys are referring to that I shared on LinkedIn, that, that it's actually now nearly a year ago and I was saying things like AI generated video is gonna get ridiculously good fast. And that I think 2 months ago Sora came out and it is ridiculously good. Yep. I think I still stand by like the lack of compute will block cheap and abundant. Like when I say it said AGI there, I meant like super intelligence until about 2030. I think we're, we're already with the GPT-4 making data centers struggle and even training of something like Sora. I think I heard someone talk about how Microsoft were having to do interstate high bandwidth links because they couldn't co-locate enough GPUs in one place without knocking out the entire grid of the state. So they had to train these models across these giant data centers. You can't put more than 10,000 H100 GPUs in the same room. Room without almost having its dedicated power plant for it. So I think energy and compute will be a challenge for at least the next 5 years. I said context will become cheap in the next 18 months. I think we've already seen that with Gemini having a 1 million token context window. Maybe give me some more updated predictions. Multimodal is going to be one of the ways that we really unlock block growth in these models. So language is amazing because it's such a compressed version of the world and that makes it really easy to learn on. So almost like these LLMs have got to skip the evolutionary phase of bouncing off the walls and grunting and they've gone straight to, it's possible to reason in language because language is a great condensed express form, condensed form of expression and reasoning. I think there's so much more information in the world and video will give us that information, except it's way lower information bits to data bits in the technical sense. But on an hour or even a minute of video, there's probably a paragraph's worth of text. You know, they say a picture speaks a thousand words, but really a picture is probably 100 tokens maybe, or like 200 tokens. And so being able to get the same amount of information from these images as videos will be interesting. But if you, when we work out how to train on these videos, there's like an abundance of like training data that we've not yet managed to unlock. And that will help take us forwards. I'm very interested to see what GPT-5 looks like and like how much of a step is it from GPT-4. And I'm hoping it's a big step. But if it's a small step, it probably means we're going to need a new architecture for reaching like these really highly capable models, which is a shame. But it does mean for the next kind of 5 years or so, things get way more predictable. And actually, we can really say, okay, these models are going to get incrementally better. How can we massively leverage what's possible incremental improvements rather than exponential improvements, which is easier to deal with as like the uncertainty of building a company and operating in this space, but is still less exciting than being like, woo, we're gonna have superintelligence very soon and all our problems in the world are solved. Yeah.

Anything more? I've got one more. I think the cost to latency to quality ratio of different models is gonna start being on top of people's minds. And I think we'll see small models running every second, for example, on your devices and bigger models trying to figure out scientific discoveries. But on our smartphones, for example, we'll have on-device agents that will try to help you out based on what— and another super exciting thing for me is with the launch of Apple Vision Pro, we'll have multimodal small LLMs, which will try to analyze everything that you see, for example. So those need to run multiple times per second. And I think that, that kind of on-device stuff and ensembles of different models, which are put together in kind of a gentle form on software like Wordware will actually be, will actually start to become more and more useful. Like right now, Google Assistant doesn't even work. I don't know. I don't know if you did turn on Gemini on your Google Assistant, but It messed up my morning routine. I used to be able to say, okay Google, and it gave me, oh, it's just turned itself on and I said good morning and gave me the news and told me what's the weather and now it doesn't work. We regress.

You just turned on everyone's Google Home who's listening.

It's like, yeah, yeah, it's listening right now, but I think it's gonna actually start working on the devices we use, not in some kind of siloed ChatGPT web website, and that's gonna be huge as well because it's gonna give people who don't know anything about AI all of that benefit without needing for them to go seek it. It'll come to them.

So on this, the hottest topic right now, and I'd like for you to— you can be on the fence, but I'd like an answer. ChatGPT versus Anthropic versus Mistral versus Gemini. Go.

You can find them all in Wordware. You wanna know my favorite models?

That's good.

That's good.

Yes.

But we, we, it's true what, what Robert said. We have a brainstorming function which combines all three of them and we make them argue and they argue in very different ways.

Wow.

That's great. That's great.

I'll let Robert finish.

So, so really you don't have to choose, but so it, it does come down to intuition of what problem are you solving? And it also comes down to do you care about the cost, quality, or latency more? And if often for prototypes you don't really care and you can just throw Opus or GPT-4, the big powerful models at the problem and it'll probably work without with minimal prompting effort. When it comes down to wanting responses to be quick, my new favorite model is Llama 3 8B. It was Mistral 7B. I'm a big fan of the Mistral models. The Mixtral, which is like the mixture of expert models from Mistral, are also excellent. But Llama 3, both the 8B and 70B models are really, really good on that cost-quality-latency trade-off. Haiku is my go-to model for if I think this needs to be quick but also needs to work very well. And it's also cool because it's multimodal and it's got a lot of context. So 200K context, you can throw in images and it will will be pretty reliable. It's better than GPT-3.5, but also cheaper and massive context. So that's my current go-to models are Llama 3 and Haiku, and then GPT-4 Turbo for heavier lifting stuff.

And this kind of answer shows you that everyone has a personal preference. It's showing you how much complexity there is in choosing one. That's why we have to give people the ability to play around with them and see what works. For poem generation better. And with one click of a button, you should be able to switch. One of my kind of underdogs that not many people are talking about is a model from Cohere, CommandR+Online. You can also try it in Wordware. It's basically, I don't know why people are not freaking out. It's one of the best scrapers, but it's better than a scraper because you don't even need to tell it where to go. If I right now write in any of your names and connect it to CommandR+Online, I would get scary amount of details about you. And I think it's mostly LinkedIn, but I don't know. And it's just sometimes somebody is like, when I do that demo are like, oh, that's not public that I'm talking to this and this company. And I'm like, oh shit, that's so scary. That's an added flavor. And yeah, just if you're using it in Wordware, just make sure you toggle the online version of it at the bottom of the choice of models. We'll work on our UX to make it perfect.

Shicker, we can use it when we have guests on the show. Scrape everything about them. I know about your child 5 years ago in the Philippines. Interesting stuff.

How did you know that? Wow. Have you been using Webflow already?

Yeah.

It's gonna go dark really quickly.

Yeah, it can, right? It can. With this shit, you gotta be really careful. So I appreciate it when folks are a tiny bit careful. We had, uh, we had somebody also from the UK. Have you heard of the startup Synthesia?

Yep.

I haven't.

Yeah, they basically generate videos that are used for, for us it was a use case of training, so we needed a bunch of training videos that you could change at any given time. And so you could prompt it using words and then they would generate the video with the same facial structure of whoever the heck gave it the data. But anyway, that was a difficult conversation to have and not be dark because there was a lot of conversations around, oh, look at this fake deepfakes. That's what it— I was looking for that word, dude. Thanks for giving that. Deepfakes was the complete— that's the first thing people said when we started asking questions like, hey, what do you think about this company? We typically do that before we have a call. We ask our friends who are software engineers, the friends who are in the space, and then we go, okay, what do you think about this company? First thing that came out was always deepfakes. So we had to completely address that on the outset and say, hey, listen, there's a lot of cool things that can come out of this, but the dangerous shit is scary. That shit is still pretty scary. But yeah, really interesting space, guys. Shikhar, any—

I think there's a whole nother podcast and you're right about, yeah, all the image generation tools, all the like face cloning. There's a world that's gonna be— out there and AI girlfriends.

It's gotten so mainstream. Yeah, it's gotten so mainstream. Just not just on the image side, but also the voice side with I don't know how many of y'all are, and he had to actually pull it down from social media because people were suing him. So it's incredible that it's already here. Now, how do we figure out what happens next?

Yeah.

It's definitely a different conversation.

I've seen Sarah Glow's Jen, how's it called? Hey Jen, hey Jen version of a video with fake voice and fake video, et cetera. And I just took 20 minutes and was able to create slightly worse version, but still I was able to create a talking head version of myself with a voice that sounds like me and using ElevenLabs and their model that's hosted on Replicate where it Just works.

Wow.

And the reason for it is because we have a chief of staff who basically wants me to record a Loom for each one of our clients. And I'm just there, dude, I can't record all these Looms. It's just like, I'm gonna steal your voice and the way you look. I'm gonna do it myself. And this is what shows a good initiative.

Andy's going to create a Wordware prompt that uses CommandR+ to learn about the clients. To write the script, to then generate the video, and it'll all happen end to end. Uh, the world's getting crazy, man.

Yeah, it is. I watched that, that TED Talk that you had generated. I was a little bit confused in the beginning and then I was like, wow, this is, it's just interesting that you can do shit like that these days.

It's v0.1. I think the next 3 to 5 years is, it's gonna be incredible. And then from 5 years onwards, when we are chatting to investors and they're like, what is the 10-year plan? And we're like, dude, I don't know what's the 10-year plan for this society. I don't know what's gonna happen to us. I'm not gonna make predictions. We're planning to, yeah, to, to help that transformation the next 5 to 7 years. But beyond that, I'm just, yeah, it's, it's a lot to understand and a lot to comprehend in your brain.

On that note, Filip and Robert, Before we close, I want to give you the stage to shout out the work that you're doing, how people can reach you, whether it is down to hiring or funding, and just want to give you the stage to shout out your team because it looks like an incredible product and you guys are crushing it.

Thank you so much. So for any listeners of Things Have Changed podcast, we've got $250 of credits on Wordware. So we just passed this. Just enjoy it. Just try all the different models. Go to wordware.ai. It's like software, but with Word and just sign up for anyone right now. The signups will be fully open and enjoy the product. As mentioned, anyone can build more complex AI agents. Make sure you go through the onboarding process. As we've mentioned, this is a new programming language, so it takes the 5, 10 minutes. Don't expect to be able to get in and start creating amazing agents. It is still somewhat complex to, to do these things. So that's one thing. We are also actively hiring. I'll let Robert do the kind of foundational engineer job description, but whoever is very interested in the community about agents, just reach out to us at founders@where.ai. We have multiple roles open and it's actually quite— in a very early stage startup, it's all kind of fluid and mixes together. But on the engineering note of hiring, Robert will say a little bit more. And lastly, we are helping a bunch of enterprises. If there are any enterprise customers listening to us, we're more than happy to give you free consultancy at the beginning and help you figure out how to, how to, for consulting enterprises, how to deliver on that ROI and all that booked revenue. And for many other people, build Copilots and workflows that will make your enterprise more efficient. So I'll switch to Robert for the hiring stuff.

Yeah, I think founding engineer-wise, there's, there's all sorts of incredible things to build. We're a full-stack TypeScript stack. We love serverless, so we love doing things as efficiently as possible, worrying about building the value-add stuff, not the undifferentiated things that everyone has to build. Basically, if you love building things, if you love making things, if you're excited about the world of agents, hit me up on Twitter, bertie_ai, or email robert@wordware.ai. And yeah, tell me why you're excited to build on Wordware. But yeah, not looking for anything specific. We're a startup. There's all sorts of roles to fill in the engineering side, but a passion for design or for solving hard problems with agents, like something that shines that you're a builder and a maker and excited about the space is what we're looking for.

Beautiful.

Amazing.

Guys, I want to mention one thing. You guys tag team so well. Where Filip takes a certain section and then Robert takes the other section, even on the message front.

So that's great. Years of practice. We've known each other 10 years.

It's, yeah, too long.

It's a beautiful partnership.

Um, yeah, no, it's, it was really fun, guys. Obviously we didn't cover all of what we wanted to cover. There's so much in this world that we're really curious about, but man, appreciate the hour and a half of just spending on a weekend. Before you have your beach day. Really appreciate that.

Thank you.

Thanks for inviting us. It's, yeah, it's been fun. It's, yeah.

Awesome.