The Accidental AI Canvas - with Steve Ruiz of tldraw

Welcome back to Latent Space. I'm very excited to have my good friend Steve Ruiz. How are you this morning?

Or hey, how's it going? Evening for me, but yeah, evening for you.

Well, thanks for staying up. I have had the good fortune of knowing you before you got famous and actually hanging out in the precise office and studio that you're recording from right now. Congrats on Make It Real. Congrats on TL;DR. I, you know, I think it's been something that's sort of years in the making, but it's fun probably looks like overnight success to a lot of people.

Yeah, thank you. I, yeah, it's kind of a funny story. I don't know, where should we jump into it?

Well, I like to people give you a little background on the person. You don't have a lot of detail on LinkedIn, just like myself. I just found out just before recording that you're also a late entrance into tech. So maybe just like, you know, what's your background coming into something like TL;DR? What makes you so unique at doing sort of creative collaborative experiences? Like, you know, I've, I know you and I've, I've, I've actually used CL Draw, so I have some appreciation for how hard this thing is.

Like you said, I kind of came into this a little late and kind of came into it from a weird angle. I, my background is actually in, in like fine art and studio art. So I have my master's from University of Chicago in visual art and would write about contemporary art and, you know, put together exhibitions and do my own paintings and drawings. And that was back when I was living in Chicago. And then when I moved over to the UK, kind of, you know, got a new studio, kept that going. But when I turned 30, I kind of decided I should probably make some, make some money and kind of work with other people closer than I was at the time. Studio art is primarily a solo thing. Um, I'd always had kind of like an analytical background or kind of side to me. My day jobs were, you know, I was working for lawyers. I was doing this writing, um, like magazines and stuff. Um, and so when I, when I did that kind of that switch back, um, eventually to, to design and to product design, um, it Uh, I was also able to use a little bit of tech, like a tiny little bit of technical skill that I had had just building like WordPress websites for myself and other artists as portfolios. Um, kind of take that and, and just some natural curiosity around the way that the products work, um, and kind of create a career direction that was more around prototyping and like technical design and doing that, um, kind of like doing the design on the bits of a product that, that really couldn't be designed otherwise. Um, so the, the interactive bits, the, the, the bits which are, um, maybe more, um, there's more questions about them. There's no clear answer in terms of like, how should this work? You know, in all those places you kind of have to build something, um, in order to, to figure out what you want to build. Um, it turns out, you know, to, to skip right to the end for a moment, like, uh, Canvas is just full of those types of problems. So it's no surprise that I ended up there. It's like kind of an extreme form of the same problem. Uh, but yeah, so I was working, this was back in like 2017, 2018. And I used at the time a product called Framer. That was back when it was more of like a code product than what it is now, which is more of like a visual builder that is kind of backed by code. And so I sort of just drilled into that. It was cool. Uber was using it. No one knew how it worked. No one could use it. So I got good at it and got a lot of advancement, early traction, whatever, in my career based on that. But it also, you know, taught me to code, taught me to think about, you know, building things that other people are going to use, uh, uh, taught me about kind of like the type of code that you write when you're in an exploratory phase rather than like in an execution, um, like production phase. And that, uh, yeah, I actually ended up working for Framer. I did their education for a year, uh, which is very different than the type of product design that I was doing before that. Um, I did a lot of video tutorials and, uh, writing and tweeting, trying to figure out some way to make technical design content interesting, you know, in, in little chunks that people could, uh, could consume. Uh, and I, I joke that like they probably got less out of me in that job than I got out of the job itself, like, because, um, Yeah, I walked away from that. Um, not sure if I'd helped anyone really learn how to use Framer, but I certainly learned how to, how to tweet. Um, I learned how to record a good GIF and learn how to talk into a microphone and all that type of stuff. Um, and so in the next roles that I had, which were, I worked for a company called Play out in New York who was also doing design tools. And I really wanted to work in design tools after that. Play's doing like a mobile Um, I guess now it's like just general iOS, macOS, um, platform-specific design tools where you're, you're using actual elements from the, um, kind of widgets from that, that, um, component or component collection. I don't know. Um, in your designs and kind of bringing that a lot closer to the end product. Um, and at the same time I started getting into open source. I'd kind of done some popular open source before. But, um, this was now 2019. It was, it was lockdown. I had a little bit more time. I also had a daughter, so not that much more time. Um, or I'm sorry, it was 2020. And, uh, yeah, I guess that open source that I started getting into started swinging back towards some of my kind of artistic interests or studio interests and kind of visual interests. Those were the parts where I felt like the problem space was really underserved and really, it wasn't necessarily like technical problems that were really hard. It was more subjective problems where I think the thing that was lacking was the taste or the opinions or the feeling for what good solutions were. So the first kind of problem like this that I got into was arrows. Like I had, you know, two boxes or two points arbitrarily placed. I want draw a good-looking arrow, like a quote mark, like good-looking arrow between the two, you know. Well, that could be anything. That's not a math problem. Maybe it involves some angles and linear geometry and vectors and all that, but it's like the good-looking part really was the, um, was just like my own taste and my own eye and like tons and tons of iterations. And arrows are super tricky and there's a million ways for this, you know, edge cases when things are overlapping or things are too far away or too close and all this. Um, But I was working on this and I was working on this in public on Twitter, recording GIFs of boxes and arrows kind of squishing together and all that. And I think people really liked that. Uh, and they liked kind of following me on this, uh, somewhat obsessive journey, which was, uh, technical, but it wasn't like, it wasn't like trying to crack an algorithm. It was like trying to, trying to figure out and identify the, the rules governing an aesthetic experience or an aesthetic you know, um, thing, which was a good looking arrow, uh, that became Perfect Arrows and that was pretty popular. But the next one really is what, what kind of broke my popularity on Twitter or just in the space. And that was a project that ended up being called Perfect Freehand. This is, um, a little hard to describe if you've ever used like an iPad pencil or drew with like a stylus in Photoshop or something. Um, like the harder you push, the thicker the line gets and the lighter you push, the thinner the line gets. It kind of is like this ink experience. Um, and that's, it's not an easy problem, but if you're doing it in a kind of a Photoshop style, like raster environment, you know, the solution's pretty straightforward. You just draw like, uh, you interpolate like tons and tons of tons of whatever shape you're drawing. In between each point that you've actually moved your mouse to. And you just change the size of that little stamp that you're making. So it's like a little circle, slightly bigger circle, slightly bigger circle, slightly bigger circle, but they're all really tightly packed together. And so it looks like a kind of a, um, a line that's changing its width as it moves. Um, my angle on this, uh, the reason why I spent so much time on it was that I wanted to do that using vectors. I wanted to get, um, a bunch of points in and then like a polygon that sort of find the outside of that shape coming out? Because I like to work in SVG and, uh, and it turned out that this was like an insanely hard problem that no one had solved. Um, and if they have solved it, they certainly didn't open source it, but I couldn't find any good example of a variable width line that actually worked fast enough and, uh, consistent enough, et cetera, for, for it to be digital ink. And so again, I did this in public, did this in on Twitter, like a million GIFs of lines that look terrible, but, you know, like slowly attracting more, uh, like getting closer to the solution, attracting more people who had solved this problem or tried to do this, or they wrote their PhD on ink. And let me tell you about, you know, how arcs work in this environment and all this stuff. Um, wow. And I, uh, yeah, it was, it was fantastic. Like I met so many good, you know, people who had like, were experts on this or, or something like it. And, um, slowly we made a, uh, an algorithm, like a really, really good tight, um, little library for, for doing exactly what I wanted. Like here are a bunch of mouse points or just arbitrary points. Like give me back a polygon that surrounds them and let me essentially draw a line around the edge of that polygon, fill it in and it'll look like ink. And, um, so that was Perfect Freehand and that's now used in like Canva uses it, like Draw.io uses it, ExcalDraw uses it. We use it at TealDraw all over the place. Um, yeah, it's, uh, it's just like a significantly better than the next best solution in that space. And there really wasn't even any known solution in that space. So someday I'm going to be checking out at a hotel and see my own ink and, you know, a little iPad or something like that. Uh, and so, um, I might as well just kind of finish the, the, the whole, uh, origin story or something, but that, that's kind of led right into TealDraw. Is that I had integrated my ink into Excalidraw and I, you know, spent time in that, that code base. And I'd also like created several like infinite canvas-like tools to help me build perfect freehand and visualize it and sort of, um, you know, do my ink and pan and zoom in and program against this thing. And so I had done, including globs.design, which I won't necessarily talk about, but it's a, um, Yeah, it's kind of like a weird experimental design tool, but anyway, it was like, it was an infinite canvas. It was very much like, you know, Framer, Figma, et cetera. And after doing Excalidraw and been working on these kinds of projects that were in the same area, I was like, you know, I, maybe there's, there's a market here for, uh, or not even a market. It was just like, I think the thing that I want to work on next is, uh, is like a general purpose. Kind of like whiteboard-like engine, uh, mostly for myself. I had built Globs, but the only thing that you could put on the canvas in Globs was a glob. So I had all this like code and these solutions that I, uh, you know, was like hanging around. It could kind of see how I would adapt it. Um, and so that's what I started doing. And that was the next story that I was kind of telling on Twitter is that like, okay, here's how selection works in something like Figma or, or something like Miro or Framer. Sketch, these, uh, these kind of heuristics, like not, um, heuristics, these sort of conventions that are part of this really complicated thing called like the infinite canvas. Um, you know, going all the way back to whatever, uh, Flash and before then, you know, Adobe Illustrator and before then all the way back. Um, and they're all pretty consistent between products. Like if you're making a canvas this way, you have to kind of do them all. Like your undo redo should work in a specific way. Your, uh, selection should work in a specific way. Like, you know, the camera position and how how the camera moves should work in a certain way. All the modifier, like Option drag to clone. And then all of those became their little vignettes of, uh, of how I was building this thing. Uh, this was now like spring of 2021. And I had, uh, everyone from any Infinite Canvas related creative product kind of like in my inbox being like, hey, can you come work for us? You know, like, let's talk, let's do this. And so I was either going to go work for Figma or Adobe. Uh, and, uh, I ended up going with Adobe in part because I think FigJam had just come out and, uh, and the, the team at, at, uh, Figma were like, well, this, this is competitive with, with FigJam. I'm like, this thing is like nothing. It's like a little open source, you know, it's like no one uses this. It's just me trying to get to 10,000 Twitter followers. Uh, but you know, it's mine. So no. So I went with Adobe and I told, but I told them, I'm like, uh, I don't want to start for 6 months. Like, this is actually a pretty fun project for me. I want to get it out of my system. Uh, you know, let me start in January and, and just work on this. And so they said yes. And I, you know, quit my, uh, working with Play, uh, and, and said, I'm gonna go work on this, this little open source thing for, for 6 months. I have some, you know, contracting money in the bank. Let's drain the, the company account and, and do this. That's not what happened. Uh, the, the, Like I went full-time from a Wednesday. On Thursday, I had a very large communications company say, hey, we're moving our whiteboard that we've designed for specific touchscreen devices. We're moving that into the browser. It turns out people want to use the whiteboard on their phones and on their laptops and all that, like, like they do with Miro. And so we need to make this thing that we wrote in C++ to be highly performant on these, you know, kind of tiny microcomputers that were part of these interactive touchscreen TVs. We have to make this work on the web and we don't think it's gonna be good enough. We have to build from scratch. We don't have the team. Can we just build on what you're building? At the time, this thing wasn't open source. It was just sort of, but it was getting there. And I'm like, uh, yeah, sure. Like, gimme like $75,000. I'll let you see the source code. Just, you know, don't, I don't wanna talk to you very often, you know, like I'm not working for you. I never wanna see your code, but you can look at mine. Um, and they said yes to that. And so I was, you know, funded for those, those first 6 months. Uh, and I got to work on this, uh, you know, without having to feel bad about it. Uh, and I'd also opened up, uh, eventually opened up TealDraw to be, um, like sponsorware that if you were sponsoring me on GitHub, you could, you could access it, you know, in its kind of primitive state, uh, on tealdraw.com. And had like a couple hundred people join that way and sponsor me. So at one point, like my sponsorship was, you know, over $5,000 a month, which is not massive money, but it's like, I wasn't doing anything different. So it was pretty good as a kind of a passive thing. Uh, anyway, I shipped it at the end of November, uh, 2021. And it was, uh, very popular. I had just open sourced everything. I was just like, You know, the TealDraw.com app, the library, the canvas, and, you know, it was organized in a certain way. Um, it just made it all public. Everything was MIT, you know, let's just, let's just throw this out into the world and see where it goes. Uh, well, it went pretty far. There was like number one on Hacker News for a while. It was like the top trending repo on GitHub. Um, a lot of people get like 40,000 people showed up at TealDraw.com to use it. on that launch date, which was all good. Like so far this was all within my same narrative of, okay, this is cool. I'll make this and then I'll go do something else afterwards. The, uh, the thing that really surprised me was how many teams wanted to build on this. And they weren't like, they weren't building whiteboards. They weren't like Miro competitors or Figma competitors. They were just like apps that wouldn't, you wouldn't expect to have infinite canvases inside of them. Uh, but, and, and they wouldn't have built it except that I had suddenly made this very easy and I had suddenly shrunk the, the development time of this like whiteboard-like feature in their product from like 3 years and 3 people to 3 weeks and 1 person. Um, and not even 1 person, just like, you know, no new developers, no new team, no new graphics experts, no, no. Computational geometry guys, like, you know, we can do this. We, the canvas itself is like React all the way down. So even if you wanted to customize it, you'd just be writing React components and then a little bit more code on top. It got like, I was totally overwhelmed by inbound from companies who were like, I wanna build this, or I wanna acquire you, or I wanna, you know, I want you to build something for me, or, you know, I want this in my app. You know, how do you help me? How can I do this? And, and people were shipping things also, like within 2 weeks, 3 weeks, like, like production ready, like people had taken this and run with it. And so the, the story that I started to get around TealDraw was that like, okay, well, this is, this is a cool little whiteboard, but it's also a kind of like filling a gap that no one knew was there, which was that like, um, in the same way that like Mapbox or Google Maps, you know, provide maps for apps that would definitely not build maps themselves. Like maps are insanely hard. Like your little local food delivery app, like wouldn't just, wouldn't have a map in it, you know, like easy. Uh, but it is a value add if they can have it in there, then absolutely it is a value add. Uh, it's just completely impractical to do themselves. And what I learned talking to folks was that like every PM had used Miro or used Figma or used, um, you know, one of these other collaborative tools. Uh, and every creative product person was like, well, this is fun. Collaboration is fun. This canvas thing is pretty cool. Like, uh, why don't we do, uh, why don't we, why can't we put our information on, uh, you know, why, why can't we put our CRM on the, the canvas or why can't we do our sales stuff here? Like we're already kind of using Miro for this. Like, why don't we give, why couldn't we give this to our customers as well? Like, why don't we build a product around this? Um, and it was just a technical no until, you know, November 24th, 2021, when suddenly it was like a technical maybe. Uh, and yeah, there was, there was absolutely demand. So hence the, this, you know, I had to call Adobe and say, no, I'm not going to come in on Monday. Like it turned out that the best possible outcome of this happened. And, uh, there's actually a company here. Uh, and then I went out and I raised a seed round from Lux in New York and Amplify in California and a whole bunch of really great angels. Um, you know, on the story of, yeah, this is, this is cool. It's good app feels good. Companies want it. And you know, by then I had, um, had almost $200,000 of sponsorship, you know, and people were just signing up and signing up because there was no way to even be a customer. Um, sponsorship, sponsorship. And, uh, but also, yeah, it's a massive—

200— you're not saying $200K a month? That's— no, no, no.

But like, I mean, I had had, uh, up to then, uh, the amount of sponsorship that I had received was around $200,000. Um, and I think some of the recurring stuff was like, uh, like $5,000 a month. But yeah, like it was, uh, which is in the top echelon.

A lot.

Yeah. Oh yeah. Like, yeah. Yeah. Yeah. Um, certainly the, uh, but yeah, the, and just the amount of like kind of validation that had come in around this was like more than, more than usual. Uh, so raised a company or raised a round, put together a team here in London and, uh, basically had just been building this, this Whiteboard SDK since then. You know, we, we reconfigured the project around, okay, we're gonna be building this not necessarily for end users, but for, for teams to use as kind of an infrastructure product, a developer product. Something closer to Mapbox. And, you know, we were making demos to kind of like show different ways that it could be used. Certainly the collaboration thing is a big one, but the fact that you could put anything on the canvas that you can put on a website just because it is a, it is a web, like all HTML, CSS all the way down. And that was going really well. It was already a good story. Uh, and then, uh, I just raised like a $2 million extension for the company while I was on the final pitch for, for that. Um, the, the Dev Day was happening at OpenAI and in the morning I woke up and I was getting all this kind of action on Twitter because a developer at Figma had used TealDraw to make this little demo where you could draw a website. Click a button and get back a big popup that had your website in there. It was like a prompt, like, uh, you know, hey, you're a developer, you just got this wireframe from your designer. Can you give it back as a single page HTML file? Um, and, uh, and it would do it and it could do it. And then you could show that website to, to whoever is using the app. Um, and we took that and we're like, wow, you could do so much more with this. Like, like with TealDraw, just like it's, it's only scratching the surface of the type of integration that you could do. So again, we had just finished the race, pressure was off a little bit. It was kind of getting towards the end of the year. I was like, all right, let's, let's just take this and have some fun. Let's make some, some viral shit. Maybe we'll get like 200 likes or something like that. Uh, and it exploded. Um, it was like, I think we're at like for this month, last 30 days, like 22 million views or something like that. It's just like, like it was Kanye West numbers. It was, it was really, really, really popular for a couple days. If you're on Twitter and at all technical, you might have just seen a ton of TealDraw stuff on your timeline or about 2 weeks ago, 3 weeks ago.

Uh, well, so yeah, that, that, that kind of brings us up almost to today. Um, you just released something 2 hours ago, which we'll talk about. Yeah. Um, but maybe, maybe this will bring a good time to bring up the, the screens. You know, for those who—

Oh yeah, yeah. Let me share. Yeah. So I—

We're recording video as well. You can jump over to the YouTube to see stuff, but this is an inherently visual podcast. So we have to show stuff on screen. Okay. The incremental thing I got from your blog post. So you did do a write-up, which thank you for that, because I actually didn't know that you did a write-up. It was just drawn out on the videos. Um, uh, this is the power of open source, right? That someone else had the idea. You weren't even focused on Dev Day. Uh, someone else had the idea, uh, and just like, you know, made it without your permission or, or, uh, you know, talking with you. And then the idea could spread back to you and you could run with it.

Yeah, exactly. And, uh, we had made a lot of the bits and pieces like in place already based on, you know, I mean, it's, it's well documented or it's documented. Uh, it's documented. Tons of examples. Tons of examples and all that. Yeah. Yeah. Um, and it's, uh, uh, it's not, I mean, it's a big library as far as an open source library goes, but it's, uh, it is, yeah, you can work with it. Um, and once this thing got popular, uh, like the, the first thing we did was create like a starter kit so that someone could take it and like run with it. Yeah. Um, so this is, this is normal TealDraw where, yeah, you can, um, you know, you could draw, you could whatever, move things around. It works if you're if you've used Figma, if you've used Miro, it's kind of, kind of familiar to that. And you can put pretty much anything on this canvas that you want, like YouTube links, etc., just because it is a— I always have to remember to, to open this, open YouTube in a incognito, in, in a incognito window so that people don't see my embarrassing interests here. So yeah, because like, because this canvas is HTML and CSS, and like divs and stuff all the way down. You can put things like YouTube videos on there. You can even make them play, um, because again, like anything you can do in a website, you can do on, on TealDraw's canvas. What's fun is because it is a canvas all the way down, you can also like draw on top, uh, and like do the kind of canvas manipulation stuff that you might do with normal shapes, but also with this type of content. Um, so that ended up becoming like a big part of why Make It Real, uh, got kind of, kind of popular. So anyway, I'll show you Make It Real now. Um, this was a, a hastily built, uh, layer on top of the, the kind of TealDraw engine, the SDK that we, we send out. And the idea here is that you can make a wireframe and we're gonna send it to, uh, GPT-4 with vision with like a prompt, like much like the original one that Sawyer Hood had come up with, which is you are a web developer. You work with designers. They give you wireframes and notes and screenshots and all sorts of stuff. Could be anything. Your job is to come back with a single page or a single HTML file that has all the styles, all the JavaScript, all the markup necessary in order to make a real working prototype based on what you've been sent. It also has, you know, slight emotional manipulation, like you love your designers and you want them to be happy. And like the better your prototype is, the happier they are and all that. Oh, in the prompts? Yeah. Yeah. Yeah. Yeah. And so again, it's open source. You can read, read the prompts. It's kind of a funny one. Um, but we do, we do some other tricks that I'll like, for example, the, um, Well, we'll have a second as I do this. So you select something like this, you click the Make Real button, and you get back a little, uh, waiting box. We've considered running ads or, uh, like in this, uh, waiting moment, but no, I don't know, it's maybe like a kind of Zelda style, like tips, you know, tips and tricks. Like here's, here's different ways you can, you can do, um, but the, uh, yeah, for example, like we also send it I mean, this is part of the joy of like a multimodal prompt is we send it the photo, which kind of looks like the same as if you had done a copy and paste thing. So like an image, uh, and you, you, you have all this functionality worked out prior, you know, like, yeah, yeah, yeah.

Like that's, that's what I find so poetic about this, that you were just ready. Yeah.

It's like, it really, I mean, it feels like we had gone off in this, like, this very, you know, as collaboration and AI and stuff was going in one direction, we kind of just went off in our own weird, like, hey, the world is really going to need a whiteboard at some point direction. And then, and then it just, they kind of met us where we were at. And then we've been able to just be like, show up, like show up on day one of this new world of possibility with like the thing that like, if I hadn't spent the last 2 years building this, I would spend the next 2 years building this. Like it is the right. Product for this type of, uh, type of feature. So anyway, they give us back a HTML, we stick it into an iframe, put that onto the canvas just like we did with that YouTube link, and I can, uh, I can interact with it. So it should be going from orange to pink, uh, orange to pink. Hey, it got it right, man! It's given us a hex code. I can click the hex code, uh, and it gives me, uh, you know, YouTube links, uh, etc., just because it is, uh, um, I always have to remember to to open this, open YouTube in a, either in something like v0 or some of these other kind of prompting environments, like the only way for you to then make this better, oh, maybe you can do this with ChatGPT or something and you could write like, oh, actually, you know, instead of, or like you missed the labels, like it should say orange and pink, you know, on top of this thing. And it doesn't. So you could go back here and like, you know, make, make sure that this is, uh, whatever, you could change the input, but because this is TealDraw and because you can draw on top of this stuff, you could, um, also, you know, write on top, like you could kind of modify this, uh, and maybe even give it the same type of markup that you would give to, um, uh, a designer, you know, a designer or something like that, you know, draw some arrows or maybe paste in a screenshot and say, hey, make it look more stylistically close to this other thing. Um, and I'll say this as well. Um, I'll say like full width, uh, for that, for that button. And, uh, anything else that we— well, let's just see what it does. Um, and then what you do is you select, you select the, the website that they gave you back, the previous result, along with all this markup. And you use that as the new input. And so that's going to give you something kind of like, yeah, it's going to give you an image that looks like this that you've now sent. But we've also kind of tweaked the prompt a little bit when you, when you do include a previous result and say like, hey, this, you know, the wireframes coming back are annotations or markup based on what you sent before. And there you go. So now we have a new prompt that sure enough, the labels are there. You know, it still works just like before. The button is full width and, and, you know, it still works just the same. So we send it back again. We send it the image, we send it the text, the prompt. We also send it all of the text items themselves separately because ChatGPT is not really great with, with recognizing text. So we say like, oh, by the way, your vision's not so good. So we've, we've ensured to have our copywriter, you know, list out all the copy that you can use. I think we even sent it back the HTML that they used for the previous result. So we just dumped like as much information as possible at this, the GPT-4 with vision. And that's how you're able to get these sort of iterative results. And the, it is it is like legitimately good. Like it feels like work. It feels like, like you're actually doing stuff when you're iterating through this way and slowly shaping and adding complexity and doing step by step, you know, as you're, you're, you're building something. And when you're done, you can, you know, copy a link to that and open that in a new tab. Like we, we host it. It's there forever. You can, you can bookmark this. If you really just needed a slider between orange and pink, well, now you have one. You, you, you know, whether you could code it or not, or you maybe not worth building or using a no-code tool to build, but, you know, we just made that in 5 minutes. If you are more on the Cody side and you want to use this as a kind of a foundation of a real project, or maybe just to like see how it like, well, how did, how did, how does that actually work? You can open it up in StackBlitz or CodeSandbox. I think tomorrow we'll have Replit. And yeah, see all the code, see what ChatGPT came up with and kind of use it or adapt it or, you know, keep it going or do whatever you want with it. But yeah, cause it is like, it is real and yeah. It made real.

Yeah. Yeah, it's interesting that you can also, I've seen some of your other demos. It looks like you're about to move us on to another.

Yeah, I'm gonna grab a couple. The ones that I have showed before, this one's a really interesting one because it is, okay, so what's on the screen now, just to kind of narrate, describe it is I have a, I have a drawing of a stop, like a kitchen timer there where you can add a minute, add a second, you know, start or stop the timer or reset the timer. And then next to it, I also have a state chart, like state machine describing the 3 states of the timer starts or stopped, running or complete. And like what each one of those buttons could, should do in terms of transitions or changing the state. I think you can hand this to pretty much any designer or developer and get back, uh, you know, a working result. Like it's fully, fully spec'd sort of, um, but what, what you do is this.

Divit Koshied might say, develop a state chart first and then, you know, plug it into Xcode.

Yeah, exactly. And what's cool this way is that you can, uh, well, let's do a couple of things in, in parallel. Uh, first thing I'm going to do is I'm going to I am just going to make a box over here and I'm going to say kitchen timer right in the middle of the box. And this is going to be the only prompt that I'm going to, I'm going to give it. I'm just going to click Make Real on just the kitchen timer box. Sometimes you see with these multimodal prompting, like, like someone will draw a calculator, like in a lot of complexity and say, you know, make this, make this real. And sure enough, you get back like a really complex full calculator. But if you did the same thing and you just said empty box, but just the word calculator, it would give you back the same thing is that it knows what a calculator looks like and it knows how it works and all that. So next, let's also give it the, just the, the user interface, like without the state chart. We'll leave the state chart out, but we'll do just the user interface and then we'll do just the state chart. You know, and say, hey, make this real. And then we'll do both the state chart and the UI. So we have 4 different prompts with 4 potential different results based on, you know, variations of the same, same input. So first off, our kitchen timer, where all we did was we sent it a box with the word kitchen timer. It has I don't know what this box is for, but we have a time. We have start, stop, and reset. I can double-click in. I can click start. Uh, it doesn't do anything. Well, what is this? Oh, whoa. This is that. If this, okay, well, if the number's there, yeah, then it'll, it'll, it'll stop. If I stop it, it goes, it stops. I can start it. It'll keep going again. Okay. And I can reset it. And there we go. The only weird thing is that it works. Uh, Yeah, it has a number input field for the number of seconds that I can type out. But yeah, you know what? In a pinch, in a pinch, I'll take it if I really needed just to count 60 seconds or something. Next, we have the input where, or the result where the input was just my drawing of a kitchen timer. I didn't tell it it was a kitchen timer. I didn't send it the words kitchen timer and I didn't tell it how it should work. but it did produce something that kind of looks the same. Uh, let's see if it works. So I'm going to click minute, second, start, reset. No. So unfortunately, uh, it did not make any, uh, working UI, although it did, you know, put the buttons in the right place or something like that.

Uh, maybe it overfocuses on the UI because you, you told it, you just, that's all you gave it, you know?

Yeah. Yeah. I mean, there is in the prompt kind of language around like, you use what you know about the way that applications work in order to sort of fill in the blanks here in terms of the behavior and all that. But let's go to the next one. This one is where we only sent it the statechart. Uh, there's also something in the prompt that says like, if it's red, it's not part of the UI. Like if it's red, then like treat that as an annotation rather than a, like a, a thing that you should, should actually make. Uh, okay. So this time it actually looks a lot like the previous one, but it does have these, um, minute-second buttons. Oh, weird. It has plus and minus for minute, minute-seconds. Uh, and it also has this like stopped state written at the bottom. So there's 4 buttons, you know, minus minute, minus second, plus minute, plus second. And then there's starts, uh, start and reset. So does it work? I can add a minute. I can also subtract a minute. All right. I can add a second. I can also. Yeah. And if I press start, we're now in the running state. Unfortunately, it's going up rather than down. And I can reset it. And okay. I'm just curious if I do give it an additional prompt here and I say like, this should. Count down, not up. And just kind of do an arrow towards the start button here. Let me see if that, uh, that'll make a real one. But, and then finally, we look at the other example, which is where we sent the state chart and the UI. We get something that looks much, much more like our user interface. The question is, does it work? Yes, it does. Perfect. I can stop it and start it.

Yep, that's a working reset.

It's wonderful. And in this case, yeah, my, my, my feedback was accepted. I went back to the one where I'd asked it to count down and not up, and it all looks the same, but now it's counting down. So I think for folks especially who have like who have worked in design and who have worked in sort of like user experience design in particular, like this should feel pretty familiar of, of kind of sketching out and trying to do your best to specify like what it is you want and see what you get back from your designers. You see what you get back from your developer. Um, but having like a, an environment in which to have that like game loop, that like iteration cycle, uh, alone and instantaneous and essentially free. Is really, really wild. And you end up spending a lot of time kind of like, not only getting into the head of the AI and sort of being like, okay, well, why are they getting confused? You know, what am I sending that is confusing? How do I send more information in order to like produce a better result? But also it really forces you to clarify your own expectations of like somewhere up here I have a drag and drop list, you know, where you can drag list items between. And like, I started working on this and started specing it out. I was like, man, this is like actually like not only really hard to, like, to produce a good result, but it's also like just really hard to describe. Is that like the failure was really on, on my end for just not knowing how to get the information in there because I didn't actually know how this thing should work. but yeah, but, you know, I could figure it out. I have an environment in which to figure that out.

It's fun. That's amazing. Uh, I'm still processing. Oh, you have a picture of it.

Oh yeah. All right. So this is, this is kind of annoying, like, uh, slightly obnoxious, but, uh, during this mat— like, cause this thing went massively popular on, on Twitter. Uh, yeah, like thousands of retweets type of level. Um, and there, there were some folks who like were subtweeting it about like, you know, get over it. It's just a wireframing or no-code tool or something like that. Uh, one guy did say like, you know, I prefer to, to wireframe like the old-fashioned way with pen and paper. And, uh, I was like, oh yeah, no, that, that works too. I like, this works with screenshots. I can just take this screenshot here of you know, that the dude posted of the, the drawing that he had made, you know, it's not even like a good photo. There's a pen, you know, across one of the screens, et cetera. But if you just give that with no other information, to like as a prompt, you get back a pretty good result. Like it loading right now, but I've done this before and yeah, like you get back, you know, just from this, like, photo of a, of a piece of paper on the, the guy's desk, you know, you have a, um, like, not, not completely arbitrary, uh, result, like working website here, um, that was, that was, that was inferred from just that picture with no other input, not, not even like titles or anything else. And of course it's like responsive and all this stuff, uh, And so the idea of like, yeah, like our tools, yes, I've, I've worked really hard to make all of our shapes, you know, really good and our arrows obsessively good and, and all this stuff. But like the fun of the infinite canvas and TealDraw in particular is that you could just dump like whatever you want onto the canvas, screenshots, text, images, other websites, uh, sticky notes, all that stuff. And, uh, the, the model, even as something that was in preview, like the very, very first sort of multimodal, uh, model, uh, can do a really good job at just taking all that stuff as the input. And, um, and yeah, like, so we accidentally made a really, really good visual multimodal prompting, uh, uh, application environments or, or user experience environment. I'm not even sure what we're going to call this It's a TL;DR.

Uh, you also had a, you know, pre-show prep. You also talked about parallel prompting. Is that basically just, uh, prompting and then moving on to something else? Is that, is that what you mean?

Yeah, that's kind of what we did up here with the, uh, the stopwatches. The fact that we could get multiple prompts going at the same time and like arrange them spatially. Uh, and, um, people have done this also with imagery, uh, to say, okay, well, here are— we're going to use DALL-E, we're going to kind of like make a tree of prompts as you go, you know, different iterations based on, um, which— whatever, you make 4 iterations, you pick your favorite one, you keep going, kind of like what you do in Midjourney. But to have that spatial and to have that like arranged on a canvas so that it actually can make sense to you and you can kind of look back and follow it, follow forward, that like I don't know, like whiteboards, infinite canvas stuff is just really good for a lot of things. Um, and, uh, so organizing like a whole bunch of different content that is, uh, irregular or ephemeral or, um, or has a kind of like ad hoc meaning configuration, like, you know, things that are next to each other or things that are in a grid or in this case, you know, just even what we have here for what we did with the stopwatch. Like there's an implicit meaning of like, okay, the source is on the left, the result is on the right, and any further iterations are further on the right, right? Like we didn't put that into a data model. We didn't structure that in any way. It doesn't actually— that meaning relationship doesn't really exist in any part of the product. It just exists to us because we— we can make sense of it. Um, and yeah, so for, for this type of thing, not only is it cool that now a model can make sense of it as well, but, uh, but yeah, for, for organizing, uh, complex iterations of imagery, complex iterations of, of outputs, etc., like, yeah, the canvas is a place. I really do believe that.

Yeah, I mean, that's really incredible. It's also, it's a little, you know, I think a few developers are kind of scared about, you know, how much of this, their jobs this does. Obviously there's a lot more that it can't do.

Yeah, so the, will this take my job story is interesting at this moment.

I mean, I'm not actually concerned, but I'm curious what— Um, I, I think this augments like, uh, my, actually my concern as a developer is that this is good but not good enough. You know, like it's good for throwaway UI. Uh, but I would, I actually export the code and take that code. Um, I don't know. Uh, it, it looks like, uh, your first MVP was just HTML files, which, you know, if it's a single HTML file, it can, it can have some JS and some CSS. Um, I saw some problems with layout, uh, in there, which I don't know how good it is at layout. Uh, it's, it looks like you could just prompt it for Tailwind if you want Tailwind. Uh, I assume it can generate React. Uh, I don't know. What, what are the limitations of this thing?

Well, there's, there's the limitations that are in that particular demo, which is that like, well, it couldn't do React because it needs to just be a single compiled thing. Uh, you can't do any multi-page stuff or, or anything like that, but that's more of like how we're structuring the, uh, the project rather than like a specific requirement of, of the project itself. The, I mean, we've talked about having, well, there's, there's two kind of things. There's one is like how big is the input window and how big is the output window or something. Uh, so in theory you could have, uh, the input be here's a entire full stack React application. Together with all my UI and all this, all my components, et cetera. And here is a screenshot that I took of the landing page where the menu is in the wrong spot, you know, and I'm going to annotate that with some arrows and some text in order to say like, here's what I, where I want it to be, or here's what I want, et cetera. And for the output to be, you know, a diff that I can apply to my codebase, like produce the, the, the, the git, like basically like produce the commit, um, that would change this and have that commit be against multiple files and, and et cetera. Um, in order to, to have potentially like a solution that is just, uh, like ready to go applicable, like a patch or a, a PR that you can make. There really isn't any limit in that. And we've seen with Copilot, et cetera, the challenge is more on the input side than the output side. Like, absolutely, you could figure out a way for this thing to spit out like a working iOS app or something like that. The question is like, how do you tell it what you want? And How do you iterate when it gets it wrong? And just doing zero-shot, zero-shot, zero-shot is like really a frustrating process. But if you do have a way of iterating, if you do have a way of kind of like step-by-step moving towards the solution that you want and kind of like, you know, getting it into like, okay, well, this is good, but it's not great, could be better, etc. That's how you actually make that type of complex output more practical or more realistic. is that you probably won't get, ever get the prompt just right. Even if you have like a really, really, really good, you know, 3 generations from now agent, like you, you still have to put that information in. Um, but you're never going to put the, all of the information in the first time. You need to be able to iterate on it. And so with visual stuff, uh, I feel like the Canvas, like what we were looking at, um, that's part of what it unlocks is that like space of iteration, that space of, uh, you have a way of marking up the result and using that as the new prompt. And that's kind of new.

Yeah, yeah. Multimodal prompting is such a brilliant concept that, you know, I think it's gonna be a norm for some things. In my mind, you demonstrated, you know, I would say coming from Photoshop, there's this concept of layers. You can kind of simulate layers in Teodraw. And I see like an emergent property of layers in this kind of prompting, which is there's the UI layer and then there's this state chart layer. And those two things seem like pretty useful in specifying a prompt. I was just wondering if you've thought a little bit about like other dimensions or other layers that would be useful in multimodal prompts?

Yeah. So, uh, one thing that we've done is to bring in screenshots of other apps. Like, here's stripe.com, like, make it look like Stripe, you know? Uh, so there's sort of like, like, or like, here's linear.com, like, let's, let's do it this way.

Uh, make my DevTools website, uh, or make, uh, you, you should just, you just you should just like give it design and ask it to make pop instead of make real. Yeah, exactly.

Make it, make it more, make it more, make more pop. Uh, so there, there's the idea of like bringing in style as like a, as another, um, another in like part of the input. Uh, flowcharts are, are absolutely useful. Uh, I mean, this is, it really just boils down to like, what would you really give a developer who you are working completely asynchronous with, you know, if you had to spec out a project and put that, print it out on paper and mail it to a developer and they were going to mail back a disk with an HTML file on it, like, what would you send if you were sending this to the moon or something? So yeah, definitely like descriptions of how the state should operate and specs on that. We've— what else have we done? We've done flowcharts, we've done screenshots of other apps. We've— I think we've, we've even just pasted in code like, like here's a whole bunch of JSON that you can use and have it just read that as the, as the input data. We can— you can point it at specific endpoints. You can say like, I want you to hit this endpoint and then display the results, you know, as, as cards or as items or something like that. And I mean, you don't even have to like wire this up. It's not like Retool or anything where you have to register that or, you know, it's not built into the tool. You just from an endpoint. Oh yeah, yeah, yeah, yeah. I'm trying to think of what a good demo endpoint would be. We could maybe, we could do one more, uh, more test. What is it like dog.co?

Is that, yeah, is a good demo. Yeah. Yeah. I've used that one.

Um, I mean, this, this might be, this might be kind of like the box with the word calculator. Like it might just know because it's probably been a bunch of tutorials.

Oh, it's in the training set. Yeah.

Yeah. But, but, uh, you know what, we'll, we'll do it anyway. We'll, uh, I'll, I'll share it and, uh, We'll try.

So for those who don't know, Dog CEO is one of those like demo APIs that you just set up just because it's not offensive and it's like a useful dog.

So you can hit Dog CEO. Yeah. I'll try and think if there's a more complicated endpoint that we could hit. Maybe while I'm doing this, if you want to comment about it.

Yeah, just give more ideas of what to prompt and what can go in. I definitely didn't think about hitting endpoints just because it's just not in any of the demos I've seen.

Yeah, but it works. Let me see, I'll have a big button down here. Show me a dog. Okay, so that's going to be our show me a dog button. This should be picture of a dog.

Oh, that's a great dog. No, thank you.

Oh yeah. Yeah. Yeah. And, and then we'll, we'll do some annotations here. We'll say like, okay, when, when this is clicked, get a new dog.

Um, yeah, this is, uh, Perfect Arrow's coming in. Yeah, exactly.

When clicked, get a new dog from dog from— I'll just paste in this. Um, yeah, and put the result in the image. Okay, so it's more of a, more of an instruction than you would normally, you know, I don't know, I Yeah, one thing that it's going to have to guess is, you know, the response format, right?

Because it could be anything.

This is true. Let's see if it works. Well, hey, all right. Well, that would work. Yeah. And let's see if it actually hit it, hit the right, the endpoint in the right way. So dog button. Yeah. Okay. It hit the right, right endpoint. JSON dog image. And then it put it in. So there you go. You have yourself a JavaScript tutorial in a box ready to go. And I think like, I mean, we probably wouldn't do this on camera, but like you can say, you know, like, like use the auth token, you know, whatever. And like, you know, go like really get, you know, real data back from this thing. Um, there's no reason why it wouldn't be able to do that. Uh, you're kind of relying on OCR. Yeah. Yeah. Well, not really because, uh, again, what inside of the prompt for this, we do give it like an array of all the text that you've put in. We say like, look, I know your vision isn't so good or you have a hard time reading text sometimes when it's small. Uh, cause what are like the input that, that you get is pretty wild. It's like, it takes this as a PNG, uh, and then it like, um, I can't do this in TealDraw, but it resizes it. It squishes it into a 512 by 512 image or something like this. And then yeah. So it's like, um, it, text especially can get kind of like chunked up, especially if it's small. So we, we send those strings separately. so that it can kind of reassemble, um, anything that it can't read, uh, right off the bat. Yeah, it's— this is a weird future that we've, uh, we found ourselves in. It's pretty cool.

It's pretty cool. Yeah, I mean, you know, one layer I automatically think of is backend, right? Like, uh, as someone who has, uh, works at AWS, I see a lot of systems diagrams like cloud diagrams, entity relationship diagrams for database schema. So I wonder if like anyone's tackled extending this to backend. And then obviously the next level from that is full-stack apps where you have backend and frontend.

Yeah. Short answer is yes. There's someone on Twitter that was using this to generate, yeah, like he was doing like flowcharts. I'm not a backend guy, so I don't actually know exactly what the output was. Um, but I believe it was like a, like a configuration script for AWS, um, that was built off of this, like, I think he just copy and pasted, uh, a diagram that he had made in TealDraw anyway and said, okay, well, let's throw this at this thing and see what it comes up with. Tweaking the prompt to say like, you, um, rather than building single page websites, you just return the JSON description of this configuration or something like that, or return a script that would set this up. You could tweak it to say like, here are like, yeah, all the entity relationships between different tables or items and tables. And give me the, give me back the SQL, you know, like, you know, initialization or something. So that would make all these tables and and set up these relationships. Uh, yeah, it's just again, like the hard part is getting that information in, but we, I don't know, pictures are really good. Uh, and yeah, if you can tell it that way, then it works.

Yeah. Yeah. Awesome. Um, you are also part of, uh, so you were one of two, I wonder what I think about multimodal viral hits in November. The other one also you had a part to play in, which is the local consistency models trend, where I think you worked with Fal.

Yeah, so we, and actually I do have something to show here. We actually have a couple of things to show here. We connected with Fal because they used TealDraw to create a demo. For, uh, for their, their, uh, LCM, right? Yeah, latency. Yeah. Um, but we took that and we, we made, uh, drawfast.tealdraw.com, which is, uh, basically you get a— these like shapes, these little draw fast shapes, um, and it puts the result, basically grabs that new image and puts it right next to it. And these are, um, these are extremely fast. So as I'm moving things, you should see the image, um, updating as well. Uh, and I think it— I think this was originally not a goat, you know, whatever. This is a, um, uh, wise princess. I don't know, I play this more with my, uh, my daughter than anything else. I like to see what this looks like, you know. And, um, oh my gosh, she, she does. And actually, we had a lot of, uh, a lot of folks on, uh, Twitter, um, being like, this is, this is not good, like, whatever. Like, because I had a video of whatever my daughter drawing and You know, she made this awesome drawing of a mermaid and we turned it into like this really anonymous crappy version of like an illustration of a mermaid. And, uh, they're like, no, no, the children's drawing is much more interesting. And I'm like, yeah, yeah, yeah. Come on. What? Who cares? Of course it is. But like, this is, uh, you know, this is, this is fun. So yeah, I do think it gets pretty wild. Animation.

Yeah. Like some kind of like you could make, uh, some kind of— it's, it's a— this is almost like stop motion film. Um, yeah. Yeah. I mean, we need to do more work on consistency, but like, this is getting there.

Yeah. Uh, it is a car driving through a busy market, marketplace. Uh, the fun is that like you end up, uh, after playing this with this for a little while, you end up like, uh, getting really into the particularities of the input, like the, uh, like what can you do with a design tool? Okay. You can move things around, right? I can grab some of these and move 'em around. Um, and like, oh yeah, there's a highlighter here too. So we could like, we could do some highlighting, you know, that'll, that'll do stuff. Uh, and then you kind of like, we, we couldn't help ourselves. We started making these like stories. So. Uh, one thing you can do is kind of click this little button and that'll make the drawing and the result kind of on top of each other. All right. Well, then I'll move on to the other one that we, uh, that we released earlier today, which is called lens.tildraw.com. So that was drawfast.tildraw.com. Uh, and again, this is probably not making a good podcast audio, but it is, uh, the image updates as soon as possible based on what the input drawing is. Um, and it is pretty hypnotic. So the the, uh, this one's a little riskier because it's live. So this, this is, uh, we took a project called Together, which is a vertically scrolling infinite drawing collaborative experience. Uh, a little bit like a chat room. As you're drawing, everything's just sort of moving up and it just disappears off the top of the screen, never to be seen again. So it's kind of just fun to play with.

Um, but by the way, one of the most magical chat experiences I ever had was with you. I think you were like with your daughter or something and I was, I was whatever, showing off together and you started writing. I started writing and we had chat on together at TaoJiao.com. I was like, what is this?

It's super cool. Like, you know, there's inevitably someone will write like, you know, where are you from? And everyone's like chiming in and talking about, yeah, you are like, this is cool. Where, you know, yeah, you can chat. Um, so what I'll describe what's on the screen now, which is basically we've, we're taking like a screenshot of the center, uh, like a square out of the center of this chaotic vertically scrolling chat experience. And we're sending that to the, to the LCM and putting back the image based on like a prompt, uh, like, you know, desert scene or busy marketplace or, uh, Futuristic cityscape or something like that. And so it is updating like, you know, 10 times a second as, uh, as we go.

But yeah, I would say like it's updating surprisingly quickly, like 10 times, 10 frames per second. Here we go.

Yeah. Yeah. Uh, no, I think it's now like to 32 milliseconds basically, uh, as you go. And, um, so if I draw like a big orange thing down here, it's going to kind of show up into the drawing. Um, maybe I'll do a big black one so you can see better. Uh, like, it just sort of becomes part of the, the input to this prompt. Um, and it is extremely hypnotic. Uh, this is again like lens.teildraw.com. Um, this isn't live, so no one should come on here and say hello, Steve, or anything like that, but it is, uh, Yeah, nobody's like— it is. Yeah, it's like this like slow-moving, um, collaborative kind of like hallucination experience, and it just never ends. I mean, yeah, I'm probably gonna be funding Fal completely for the next, uh, you know, their Series A or something like that. But the, uh, yeah, are you on here, uh, Sean? I'm not, but maybe I should. Okay. Well, whatever. Yeah.

I have like, I don't know, I have a healthy respect for like the amount of processing that must be going on behind these things.

I just— Yeah. What's funny is that— Oh, cool. Perfect. Someone's doing the— He looks like—

I can't really draw. I normally draw in English. You blobbed it somehow.

Yeah, it's, everything's a little bit bigger. Um, when there's 36 people, uh, it's, it's kind of slow as well. So, uh, yeah, we, um, what's funny is that like, yeah, we're using like Cloudflare workers to do the, uh, the, the updates and the CRDTs to do the collaboration and, you know, all this like whatever, LCM models to populate this image or create this image. But there's also a laptop in my living room right now that is doing the actual screenshotting and sending that up. And so there's a big note that I had to write, you know, for my family to say like, don't turn off this laptop, don't close this laptop because this needs to be on in order for this thing to work. Yeah. So, you know, we'll, no matter how good our tech stack gets, we'll always come back to, uh, some, some laptop stuck in the corner that can't possibly be turned off. Uh, yeah.

I've heard of major businesses being run that way. Uh, yeah, exactly.

There's a Raspberry Pi in the closet.

Yeah. I, I, you know, it's weird because it's really funny because like, you know, you are inventing your own art form. Uh, this is, uh, fine art. You know, going back to your degree. Um, it's just a different—

you know, it's funny because like the output of this, like while it is like a visual output, the output like doesn't actually matter. Like it's gone in 16 milliseconds and it's not coming back. Uh, and I think with all this AI stuff right now, just where we are with it and just how completely unknown it is in terms of like, where is this useful? Uh, like the best thing that you can get out of this is like the experience. Uh, and so I think of this much more as like, you know, the thing that people will walk away from, from playing with like lens.tealtral.com, uh, should be more of like that experience of, of having interacted with this thing or interacted with it, you know, among with others, uh, rather than like Oh, there's— it made my favorite image or something like that. So I think the, uh, yeah, I don't know, as a, as a former image maker, like the idea of having, having like an aesthetic experience where the image is a major part, but it's, it's not necessarily like the, the important part, uh, or any one of these images isn't the important part. I don't know. There's some like like there's some, uh, there's something, something new feeling about this, kind of fun. Certainly I wish I was, uh, could do a big critique with all the new media artist people, um, about this and about like what, you know, where does this, where does this fit into the sort of the, uh, yeah, yeah, like other people.

Well, that's for, that's for them to, to write and, you know, for you to build. And I think that's a, you know, I would encourage you to keep building there because you're definitely not done exploring with your explorations. Thank you. Cool. Yeah. Well, so I can sort of round it out by sort of looking towards the future. You hinted a little bit you are working towards TealDraw 2.0. So first of all, actually, so it seems like you're very focused on the core mission of Canvas. And the AI stuff is a side project for now. Why not pursue it as like a full— why not pivot and like be an AI company, right?

Like that's—

I'm sure you've had a lot of those questions.

Yeah. I mean, when you, when you get something as viral as TealDraw got, like, I think I've talked to everyone, certainly every, every investor and Um, okay. So I guess the way that, the way that, yes, we, we probably could on for something like Together or that DrawFast thing, uh, make a tiny little SaaS app, you know, give me $10 a month, play with this thing. And, uh, you know, could make it, make it good. We could go in that direction. Um, there's not much of a moat. Around any of this stuff. And we're seeing that just in, you know, I don't know, Gemini is going to come out in a couple of days, weeks, or whatever. And if it's better, then people are just going to use that until the next better thing comes along. Like, it's not, there's not a lot of like unique, uh, it's not, there's nothing really defensible about like, hey, it's an, it's a drawing app plus an LCM-like model, uh, because there's going to be a lot of those models and there's going to be a lot of drawing apps. The, the thing that I think is really unique for TealDraw, the thing that we have added that is not easily created is the canvas itself. Is that like this, uh, you know, web-based, uh, hackable, extendable, uh, super refined, um, interactions and, and all that stuff. Like, you know, all the thousand table stakes features that, that drive people nuts when building something like this. Like they're all there. They're all good. Day one, you could build a really great experience, uh, whether it's AI-driven or not, like using TealDraw, um, in a way that it's just not practical to do if you're building it yourself. And especially if you're not doing like graphic stuff, there's really not that much else out there, uh, oriented towards, towards this type of thing. Um, and I think in a world where like these type of experiences are gonna— or not these experiences, these types of AI-driven capabilities are just going to keep coming out faster and faster. And like, you know, I don't know, next, next year is going to be wild. Like every month there's going to be some new, uh, you know, capability or something. The, the thing that I would want to see, both just me as a, as a person and as me as having built the business that I've built, uh, is for TealDraw to sort of become the, the place where some of this prompting, some of these ideas are explored. Um, even if we decided to, okay, we're just going to close everything up, we're going to build a product based on this, that is, maybe it's a great product, but it's, it would only be one kind of one direction, one ray kind of into this infinite space of possibility. Uh, and that, that could be successful. Good. But like the idea of building the, um, I mean, we've built the sort of the direct manipulation, manipulation core, but there are so many, even like AI-specific APIs that we could build around Tildraw for having, you know, like a virtual collaborator, uh, or, or, or working with images in a, in a more rich way. Like there's just. So much that we could build in order to make this the best possible place to explore, not just one direction, but like, you know, many, many, many directions. Um, and I think that, that narrative gets me out of bed in the morning in a much more, uh, that, that gets me much more excited. Um, and I think we're also just like the team that we have and the, the tech that we have and the skills that we have, we're more of the team to build that rather than like to, become like a SaaS product company. I'm not saying we'll never do like a, you know, pay us $10 a month and we can, you can play with our magic toy. But primarily, my goal is to make Tildraw the, either the place to explore these different models, or you might, you might think of it as like the battleground on which the winners will be kind of identified. Like right now we're using OpenAI for the Make Real thing. Maybe next week we'll be using Gemini. And now it's, now it's a question of, okay, well now we have an environment in which to compare these two models with the same input and a very advanced form of input. But yeah, like let's see which one does better. Now nothing would make me happier than to be sort of like the battlefield for multimodal prompting and multimodal AI experience.

I should also shout out Baklava as the open source vision and multimodal model. Sick. Yeah, I mean, like, so I fully understand you want to own the light cone of multimodal prompting. I think that'll probably be the title of the episode. I like it. What's coming up for TealDraw 2.0?

So really the TealDraw that you've are using now and that I'm using are, is basically 2.0. It's been in pre-release for a long time. Uh, really the only change that's going to happen once we launch it is, uh, we're going to start selling commercial licenses for it. So if you are using TealDraw in a commercial product, or if you want to, then, um, you know, if it's, if you're funded or if you have revenue, then, you know, you'll buy a license and I'll add you to our special list of customers. Um, so yeah, it's mostly just go-to-market and the necessary changes are on there. There will be some kind of fun changes, secret saucy changes that launch, but nothing substantial, nothing breaking. Uh, we've put a lot of effort in the last, like, it's crazy that we've only had it open source since May of this year, uh, this, this new version, right? And yeah, it's we've been very busy since then, but it is, uh, it's stable, it's robust. We put it through, you know, a lot of usage and caught a lot of the, the issues. So it's absolutely ready to go. Um, but I have, uh, one or two conversations with my lawyer before we, uh, we turn, turn over the license and start, start moving it that way.

Gotcha. Um, and then, uh, maybe, uh, if I could get your commentary, uh, before we, we close on, on, um, just the, the competition out there, like, uh, So you are not, you are not the only sort of canvas tool. I think I get now that I was going to ask about like Figma, FigJam, and like they have some AI thing that they're also doing. Adobe is also working on similar things. Canva is also working on similar things, but they're all individual point solutions, whereas you're more the open source, uh, sort of, uh, canvas to power all of them. Um, yeah, I feel like it's just like Scaladraw. That's like the other alternative.

that remains. I think Excalidraw, and I like Excalidraw a lot. I contributed there and, you know, we retweet each other and tease each other on Twitter and stuff. Uh, and, uh, early on I was copying features from them. Now they're copying features from me. So I, you know, uh, but no, it's, uh, it's like the collaboration space is so, has so many dominant players like, um, that I, uh, I think me and Excalidraw are tiny within that. Um, yeah, well, I, there's two things. One is that we made this very strange bet on using a kind of a web canvas that our canvas is not like an HTML element or HTML canvas element. It's like normal React components all the way down. So if you wanted to add something interactive and have that participate in the sort of of space of the canvas. Uh, the way that we were doing our, our, um, iframes and kind of like being able to write on top of an iframe, you can't do that in Excalidraw. You can't do that anywhere. Um, that is like a very strange tech choice that we made around, uh, TealDraw that is, you know, finding its home in, uh, in a few different ways. Most of the people who pick TealDraw and approach me, like the inbound that I get are folks for whom that's like the killer feature, um, to be able to, to put interactive widgets on the canvas using just React. Uh, yeah, the, yeah, I, I'll just kind of repeat the same point that you, you, you kind of suggested, which is that like, no matter how good Figma's, uh, like AI solution is, and I hope it's great because I love Figma and I use it. Um, It's not gonna solve every possible problem in this space. It's not even gonna like touch, you know, like, like you can't, like none of these things. And I mean, I already had identified like, okay, uh, there was a point where like any Kanban board was like, uh, was Trello, right? When you, when you talked about Kanban boards, you were talking about Trello. Kanban boards are in every productivity app now. Uh, I think the same thing is gonna happen with collaborative whiteboards. It's like people like 'em. Uh, I'm making it easy. Uh, people are already doing it even without TealDraw when it's hard. Like, uh, like, yeah, that's gonna become a kind of a commodity user experience in, in a lot of different products. Um, probably, you know, give me a diagram from a text prompt. Like, yeah, that is probably gonna be a commodity too. Give me an image from a text prompt. Like, yeah, that's just gonna be everywhere. We're just gonna assume that that's, you know, it's like adding a GIF to a, to a chat or something that there's no moat there. Um, I do hope that Figma has an amazing AI integration, but I think the thing that it will help you do is use Figma. Um, like it, like generating an image won't be super useful, but like generating it, uh, extended autocomplete, this, uh, this, uh, design absolutely would be. And, and I hope that they launch something amazing there. Uh, but yeah, there's, uh, I, like I said, there's just a million different directions that this stuff could go in. Uh, the canvas is just like a input device. It's, it, it allows a certain type of user experience. Um, and that's certainly not limited to design. That's not limited to whiteboarding. It's not limited to collaboration or anything like that. So, um, yeah, my hope is that there are those like kind of 10,000 products that that could be made with what we're making.

Yeah. Yeah, that's a really great mission and I see why you're so passionate about it. You're the right team for it. Okay, you know, a couple of lightning round questions. Sure. One, which is like, if you had some AI capability that you would wish for that you don't have yet, what would it be?

Oh, that's a really good question.

Helps people to do some research.

I think probably related to, it's not quite a CRM, but like a human, just normal relationship management. This is something that I've never had a problem with until I had a startup actually, where there's just a lot more people involved in my life. And it's it's hard to keep up with them all. And I think this is probably something that like an EA like kind of does of saying like, hey, there's a birthday coming up or something like that. Um, but also just, you know, identifying opportunities to, to work together, uh, to, to connect or, you know, who's an expert on this thing that I'm working on. Like that doesn't always occur to me. Um, and I think there could be, I don't know. I think the, the value of your network, um, that even if you're good at that, you're probably only scratching the surface of like, you know, how, how you could be helping the people around you and how they could be helping you based on like the specific context of like what you're working on and the problems on your table today.

Yeah, I've often wanted to build a CRM on top of Twitter, uh, because you have all the info there about what people working on, your past conversations with each other and your shared interests, you know, like it should be able to search, you should be like at bare minimum search it, but to proactively suggest is the next layer. And I guess AI chief of staff, AI executive assistant.

Exactly. Yeah. Something like that.

Yeah. Understood. Yeah. Yeah. I think like some people are working on that, but the problem is so big that they're working on like the automation piece. So like Lindy I had at my conference where they're like It's a virtual assistant you can trigger on your desktop or via email and it mostly deals with scheduling but also helps you do a little bit of research. So yeah, I think the agents field will progress there. We might take 10 years to do it.

Yeah. Yeah, I can wait.

It's all good. And then finally advice for founders, like what has helped you the most as a founder? Founder, you know, you're 2 years into your journey.

Yeah, I think the, uh, so this, this kind of comes a little bit out of, uh, like what you learn in art school, uh, type of thing. Um, but yeah, uh, but one thing is that, that basically like, uh, when you're a studio artist or you're in a studio or whatever, there's no external constraints. You just kind of are running on, well, what do I feel like working on? and the further you get away from like, what do I feel like working on, kind of like the worse your work becomes. So having like a really good, uh, feeling for that sort of desire and being able to respect and follow that, that desire as like a, um, because it's not arbitrary. Is that like, if, if you really, really feel like working on, um, on, on a thing like like, it, that might be the sort of the tip of a very complex, uh, iceberg of analysis of like the field or like what people are talking about or something, uh, that you directions and market or something like that. Like, I don't know. I think with, with TealDraw and with, as, as a founder on this, um, the thing that I've tried to do and I've tried to preserve is like the, uh, being able to prioritize based on like, what is most interesting right now. And that is, that is true for what code we write and like what features we work on. That's true for like which partners we, you know, we spend time with in terms of who's using TealDraw, the types of problems that we want to solve, like using your own sort of sense of what's interesting as a filter and what you want to work on, like what sounds like a fun thing to work on right now as a filter is not it's not naive. And it can be kind of part of your, your secret sauce. And I think it's— I think a lot of early founders are encouraged against that, and to be working backwards from a certain outcome and all that. And yeah, you do have to have to do that. You have to put that into the, into the mix as well. But, you know, be sure that you're picking the best parts out of that mix. I don't know the parts that you want to work on. Yeah.

Well, I mean, what's the point of doing this if you don't have, uh, have some fun indulging curiosity?

The worst case, you'll build something that you love. Yeah, exactly. And good things can come out. Good things can absolutely come out of like, uh, chasing an 8,000% increase in your followers or something. Dude, not— should I put this on camera? I'll share my screen. We'll look at my, uh, my Twitter analytics.

It's on your Substack. Yeah.

Oh yeah, it is. It is. I need to update that image. I've done it once already. Uh, yeah, if you, if you're a, if you're a Substack reader, the TealDraw Substack, 72 hours into this big Make Real virality explosion, I sat down and wrote a blog post and I, uh, I wanted to at least capture that, that vibe, um, of what it felt like in the middle of that, that hurricane. Uh, yeah. So it's, it's a pretty fun one. It's good to read back.

Well, I'm sure it's not the last time we'll see you do something crazy viral. I'm sure that a lot of people will be exploring TLDraw. I hope a lot of people— honestly, one thing I'm thinking about is embedding TLDraw into my input box. Why can't TLDraw be part of the input?

Hey, I'm talking to the good folks over at OpenAI tomorrow. Fingers crossed, maybe we get it inside of a— ChatGPT or something. Cause yeah, like, I don't know. I want to, I need it any faster. Yeah. Like what you want to like take a drawing or take a photo and then annotate it or like, you know, just like something else you should be able to do.

Yeah, exactly. Yeah.

It's just a good, it's just a good thing. Uh, yeah. The people cry out for it. I, uh, Failed it fast enough.

Yeah. Well, thank you for inspiring the rest of us. Thank you for everything. And I'm sure we'll hear more from you over the next few years. So thanks. Thanks for your time.

Awesome. Thank you for your time.