He Rejected Apple Twice Before He Created Midjourney

The Pope wearing a dripped out Balenciaga jacket. French President Emmanuel Macron running away from protesters. Donald Trump being wrestled to the ground by cops. We've all seen these images and we know they're fake. But can you imagine a computer generating something like this even five years ago? Everyone is talking about AI and what it means for our society. Sam Altman is testifying in front of Congress, but not enough people are talking about what I think is one of the most interesting AI companies in the world. More midjourney. It's become by far the best way to create AI generated images. And it's also one of the most fascinating stories in the history of Silicon Valley. David Hulz grew midjourney from nothing into a real business that likely makes tens of millions in profit in about a year. The way he did it holds so many interesting lessons for the future of tech and entrepreneurship. So let's dive in. Midjourney has become by far the best way to make AI generated images. These posters behind me were actually generated by midjourney. The company makes tons of money even though the team is really small. And as I'm recording this, there are 18 million people in their discord. In a way, it's crazy how quickly this has all happened, but this overnight success story actually took years to happen. Before David Holes became one of the most important leaders in AI, he ran a company that never really took off. And long before then, he was a curious kid from South Florida. David's dad was a dentist, but not your average dentist. His office was on a boat and he would sail around the Caribbean and South America where he'd pull up to port and do dental work for people. Once David came along, his dad settled down and rented out an office near Fort Lauderdale. Because of his job, David's dad had a computer. Of course, David quickly became obsessed. He learned how to program in this really old language called scheme. He even started hacking computer games. One of his favorites was this old DOS game called Star Dark Forces. I actually remember this game you played as this guy named Kyle Katarn who discovers this top secret Galactic Empire project. I was obsessed with this stuff as a kid and used to read tons of these Star wars spinoff books. Anyway, David hacked the game so that Kyle could shoot rockets out of his hands and stuff like that. Basically, he modded it. This is such a classic trajectory for people who get into tech. Lots of time on the computer, gaming, little bits of programming, and then more serious projects later on. David applied the skills he learned really early on. He ran a design business While still in high school, he attended tons of science fairs and even ended up judging a few of them. So it probably didn't come as a surprise to any of David's friends or family when he decided to study physics in college. David really admired the great physicists, people like Einstein, Newton, and Feynman. As he spent more time in college, he also took lots of math courses. He ended up graduating with degrees in both and then started a PhD in applied math. David was clearly super smart, and people were taking notice. While he was doing his PhD, he started working as a graduate researcher at the Max Planck Institute, named after the famous theoretical physicist. He also worked for NASA at their Langley campus. He was doing really cool stuff. Mapping rat brains, atmospheric science, even working on LiDAR. But at a certain point, David became worried that he was getting locked into a career path where he would just do research, write papers, and then nothing really happened. He wanted to make a bigger impact. He wanted something that could become his sole focus. So David did what lots of super smart, slightly aimless people he started a company. The company's first name was ocuspec, but that wasn't a great name. So David and his co founder soon changed it to Leap Motion. The idea for Leap Motion's main product actually came to David back in middle school. He was in a typing class, and he must have been bored because the teacher told him if he could Type More than 60 words a minute without looking at the keyboard, he didn't have to do the class. So David learned how to type really fast, got out of the class, and then all of a sudden he had time to work on other projects. That experience of learning how to type fast made David realize that there was a problem with how people interacted with technology. I know most middle school students never think about stuff like this. Basically, he thought that there was a bottleneck. People's minds worked really fast, but seamlessly transmitting that information was difficult. Cost, size, and speed are all important factors, but. But for David, the most important thing was the interaction model. The end result of that thinking was the leap, a small device you'd place in front of your computer that would track the motion of your hands and then translate those gestures into commands. But well before the LEAP was released, in fact, even before David founded Leap Motion, he had an interaction that foreshadowed the debate we're having today about AI. Leap Motion kind of invented a lot of gestural interface technology, but people weren't really ready. The iPad had only just been released, and the Windows operating system didn't yet work with touch screens at all. People didn't really think about touch beyond their phones and the trackpad on their laptops. Of course, today, touch is everywhere. And Apple recently announced the Vision Pro, which includes hand tracking as a core feature. But it's taken more than 10 years after Leap Motion's first product to get us here. Because Leap Motion was trying to do something new, David was very focused on user interaction. One of the people he talked to early on was a professor named Alan Kay. Alan is a legendary computer scientist who basically invented both object oriented programming and the graphical user interface. And he said something which crystallized the problem. David was trying to don't think about features, think about how the user learns. In many ways, Leap Motion never really solved that problem. They tried to introduce a new technological paradigm before people were truly ready for it. But today, user education, both in terms of people learning how to use the technology and also how people train the algorithm, is a big part of why midjourney is so amazing. Leap Motion had a promising first few years. Their seed round in 2011 received investments from Andreessen Horowitz and Founders Fund. But not every VC knew what Leap Motion was trying to do. There's a great story from when David was first pitching investors. He's in a room full of VCs. They're all wearing 3D glasses. David pulls a mountain out of the wall, rotates it and pushes it back into the wall. It was kind of like Minority Report at that. The lead investor took off his pair of glasses, stepped forward and said with total confidence, he I get it. After a rough week, this was like music to David's ears. I just have one question, the VC said. Can you also track things in two dimensions? David responded that because 3D is more complex, the device could definitely handle working in 2D. The VC nodded and then asked David, have you ever considered giving the first two dimensions away for free and then charging for the third dimension? David said that no, he hadn't, but that it was a really interesting idea and one that he promised to think about. They never spoke again. This story is interesting because it shows that bad VCs don't bother to think about what a technology can really do, and instead they just jump right ahead to how much money it can make, all while destroying the core value proposition. But it's also a story about people misunderstanding the kind of problems David was facing. He didn't have a monetization problem. He was trying to solve novel hardware problems. It's possible that experience soured David on VCs in general. I'll go into this in more detail later on, but Midjourney hasn't raised a cent of external funding. It's totally bootstrapped. Lead Motion, however, followed a more conventional path. After the successful seed funding round, they raised $12.75 million in Series A funding in 2012, and then a $30 million Series B round in 2013 led by founders Fund that gave them the money to release the leap. In mid-2013, it sold about 500,000 units. Apparently that was below expectations, but I think it's actually pretty solid for a new hardware company selling a totally new type of product right out of the gate. But there were issues. Tech support was tricky, user retention wasn't great, and that sales figure was probably bumped up by partnerships with other companies in addition to direct sales. Still, Leap Motion was doing well enough that Apple was interested in acquiring them. But David, the founder and cto, wasn't interested. In fact, he apparently told Apple's representatives that their company was no longer innovative and even praised Android. Needless to say, the deal didn't go through. But still, there's an interesting alternative history here. At the time, Apple actually acquired a few companies like Prime Sense that had technology that will end up being used in the Vision Pro. Had Apple bought Leap Motion around the same time, who knows how that would have affected the creation of Midjourney, or for that matter, the Vision Pro. I remember there being some buzz about Leap Motion around this time. People thought that they were building a cool new technology and they were interested to see where it would go. The company sent about 12,000 of its devices to developers who were interested in building apps that used hand tracking technology. By now, Leap Motion had pivoted away from the desktop input of their first product and toward virtual reality. But it wasn't obvious how their tech would be integrated into existing VR products. There was an external kit you could add on to the Oculus which would let you use the Leap Motion controller, but that just added a layer of complexity to a device which to be honest, people were trying to wrap their heads around anyway. In 2014, Facebook bought Oculus. After that, the VR industry sort of cooled down a bit. But in 2018, a few years after their first attempt, which went so badly that David ended up basically insulting them, Apple came back and made another attempt to buy Leap Motion. For to buy Leap Motion this time was only around 30 to $50 million, which was just a fraction of the $306 million valuation. After the Series B round, it would have been A massive drop in valuation, investors would have only been getting cents on the dollar, and the founders and employees probably would have received basically nothing. Despite all that, the deal almost happened. It even got to the point where Apple sent out offer letters to Leap Motion employees who were happy that they'd get the chance to keep working on cool tech from inside one of the world's biggest companies. But it fell through, and we never got that version of history. Around this time, Leap Motion, which Now had around 100 employees, built and open sourced their augmented reality headset Northstar, underpinned by their hand tracking technology. The decision to open source the North Star was an interesting attempt to overcome some of the problems that have plagued VR headsets. Basically, VR is really hard. It's a combination of software and hardware. The technology has to be dynamic and responsive to user experience. The headset has to be comfortable, and there have to be enough interesting apps for people to actually want to use them. When you look at how long it's taken for Apple to develop the Vision Pro and how expensive expensive it'll be on release, you get a sense of how tricky VR headsets really are. So David's decision to open source the development was really smart, and I think Midjourney is a perfect example of how that commitment to collaborative, peer driven development can work. But the North Star didn't really go anywhere under Leap Motion. And frankly, neither was Leap Motion as a whole. It was never a bad company. It was a good company with really interesting and novel technology behind it, but it just never took off the product market fit was just never there. When it came to the Leap, people weren't quite ready for the tech. And when it came to the North Star, VR headset technology just wasn't quite mature enough to be built outside of a massive tech company like Facebook. If the market isn't ready for your product, no matter how good it is, you're in trouble. In May of 2019, Leap Motion was bought by a company called Ultra Haptics. The financials weren't great. It was a $30 million deal, way down from the peak valuation. So employees with stock probably got wiped out. But from a technological perspective, it made a lot of sense. Ultra Haptics had been working with David and Leap Motion for about six years by this point, and they worked on interesting technology. Ultra Haptics makes virtual control solutions that let someone feel an object in their hands when they interact with it in virtual reality without needing to use gloves. Today, Leap Motion is called Ultraleap, and it's unclear where their Hand control sensing technology is headed, but it's growing. There are some cars that let you turn up the volume on your sound system by spinning your hand, but in general, people aren't using their hands to interact with computers just yet. David ran Leap motion for almost 10 years. He got a ton of experience building new technology, raising money, and growing a company from nothing into something that Apple tried to buy twice. It was time for the next adventure. He set up a studio to explore new opportunities. He told himself to wait until the right one came up. As it turned out, he wouldn't have to wait very long. David Holes chose to become an entrepreneur instead of an academic because he wanted to make a big impact. So this next part is kind of ironic because an academic paper was about to change his life. In June of 2023, Berkeley researchers Jonathan Ho, A.J. jain and Peter Abbeel published their research into how diffusion models can create images. David had enough technical knowledge to understand that diffusion models were about to increase the quality of AI images by an order of magnitude. He realized that the opportunity he'd been waiting for had arrived. The first version of OpenAI's DALL E model proved him right, but in reverse. Today, Dall? E2 is one of the leading AI image generators. But when the first one was released In January of 2021, not that many people actually paid attention. Dall? E One was the first text to image model based on diffusion models that existed. It was a step change from everything that came before. The Dall E1 launch also included a package called Clip which could evaluate how well an image aligned with text inputs. Plenty of AI researchers had tried to build image generators before, but using older methods like generative adversarial networks and variational autoencoders. People who were paying close attention like David was, knew that this was a turning point. Sam Altman, the head of OpenAI, wasn't exactly sitting around either. Dall E2 was released in April of 2022. Stable diffusion followed in August and Midjourney entered open beta in July. David Hulz was actually experimenting with the the preliminary midjourney model as far back as January, before Dall? E2 dropped doing things like creating vaporwave monkeys in the midjourney discord. The discord went public a couple months later. This was actually a really important milestone because it allowed David and his team to go really hard on user testing. For example, one model took about 20 minutes to create a really high quality image, while another took about only 15 seconds but produced worse images. The first approach was really fast, but it sacrificed quality. The second approach resulted in amazing images, but users would often get bored. In the end, David found a great midpoint. Today, it takes Midjourney about 60 seconds to generate an image, and it shows you the image as it's generating. There were lots of interesting trade offs like this. The open beta landed on July 12. It blew up just a few weeks later. John Oliver did a segment about it.

Roast Beef Superhero. Now, what does that look like? I have no idea. But here's what Midjourneys AI created based on that phrase. And I have to say, well played, robot.

The crazy thing is that even since that segment aired less than a year ago, the images you can create using the same exact prompt have gotten way better. Around the same time, an image generated by Midjourney won the Colorado State Fairs contest for emerging Digital Artists. It's a really beautiful image and it's a cool proof of concept for the Midjourney Discord. The guy who won the prize was invited to the Discord in summer of 2022. He became obsessed right away. He was making hundreds of images, sometimes in collaboration with other users, and was always amazed at what Midjourney could do. After he won, the winning image ended up on Twitter. The reaction was furious. People said he'd plagiarized the work of real artists. Someone even said they were watching the death of artistry unfold right before their eyes. There were people who defended him, but they were in the minority. I totally understand that lots of people feel like Midjourney is letting people take credit for stuff they didn't create and that it's taking opportunities away from quote, unquote, real artists. But I think that's not quite the right way to look at it. David says he views artists as customers of Midjourney, not competitors. Artists are using Midjourney to prototype concepts to show clients before they start work themselves. That's really cool, and hopefully it'll mean more people commission work from real artists. Of course, by 2023, Midjourney was making super realistic images like those ones of the Pope and Trump. And that's kicked off another debate about the potential of Midjourney and other generative AI models to contribute to misinformation. That pretty much brings us up to the present day. But there's so much more about Midjourney, both as a product and a business that makes it such a fascinating case study. So let's explore that in detail. The hardest thing about building an AI company today is the intense public scrutiny. You've got Sam Altman in front of Congress and lots of people who literally believe that programs like ChatGPT should be illegal. But despite that pressure, David's built Midjourney into the best AI image generator around. And he's done it by implementing four distinct strategies. Number one, harnessing the power of Discord. You can measure the growth of most companies by looking at their revenues, but the best way to measure the growth of Midjourney is by looking at their Discord. If you go on sites like Reddit, you'll see the same question pop up again and again. Why did the Midjourney team decide to wrap their product inside a Discord server? It's a fair question. Frankly, lots of tech CEOs would be too skittish to ever do something like that. But David immediately understood that Discord could be an amazing feedback loop. He believed that one of the key drivers to Midjourney's massive growth would be virality, and that one of the best ways to create virality is volume. Today, Midjourney is the world's best biggest Discord server by far. It hit 2 million subscribers in mid September 2022, 4 million by the second week of November. By April, it hit 15 million. And as I'm recording this, there are more than 18 million members. That's exponential growth. The more people in the Discord, the more people creating amazing images, and the more amazing images you see, then the more likely you are to want to join the Discord. Discord just makes sense. It's flexible and functional. It has a great mobile app, and it's collaborative. One of the biggest threats to the growth of a generative AI model is that it's actually really hard to come up with a good prompt, especially by yourself. People aren't normally very creative by themselves. They might ask Midjourney to do something like create an image of a dog. And after some encouragement, they might put in terms like fluffy dog or big fluffy dog. But on the Midjourney Discord, you can literally ask 18 million other people for ideas, so you end up with cooler prompts and you're more likely to keep using Midjourney. The Discord also acts as a sounding board and decision making forum. The moderators hold town halls where people can advocate for the inclusion or exclusion of certain words or phrases. There are tiers and some words can only be used for private images. And it's where the rules are made and enforced. Users aren't allowed to create deepfakes, not safe for work, content is generally banned, and you're not allowed to create images of controversial political figures like Xi Jinping. Given how much scrutiny there is about AI at the moment, it's a really smart idea to have Midjourney's entire global user base in one place. Discord also helps train the Midjourney model. When you use a model like stable Diffusion, you don't have a chance to provide feedback to the model trainer. With Midjourney, you proactively select from four different images to confirm which is the best one, and the model retains that data. The unique way that Midjourney harnesses the power of Discord is proof that David was listening closely to what Professor Alan Kay told him. Just focus on how the user learns. 2. Product Market Fit like lots of people in tech, David has a big bias for action. He has a really interesting quote about Many of my brilliant friends are paralyzed by their intellects, thinking overwhelms doing. Many of my successful friends are less reflective but quick to action. They are always focused on moving forward, making as many probably correct actions as possible a winning strategy. This quote reminds me of his struggles about whether or not to stay in academia. He probably met lots of people like that, super smart people who needed things to be perfect. But David realized he was different. And that difference, that bias to action is really important in AI. Even by the standards of software, generative AI is moving insanely fast. Like I said, we're already at V5 of Midjourney. There isn't time to hone something to perfection. Instead, David releases a minimum viable product, uses Midjourney's massive super engaged user base to make it better, and keeps iterating it's modern software development at top speed. And David is taking a very different approach from other AI companies. Lots of AI companies are fighting right now to become an infrastructure layer and don't want to focus on building a consumer product. They want to build a platform. You know what that reminds me of? It reminds me of when David, when he was still at Leap Motion, sent thousands of hand tracking devices to developers so they could develop apps. And it reminds me of these VR headsets that have really amazing hardware but no killer app. Everyone wants to create a platform because that's where the big monetization opportunities are. But if you don't create a great product, then the platform doesn't matter. Leap Motion never really took off because despite having cool tech, it didn't give enough people a reason to use it. Midjourney is different. Artists use it to improve their practice, but mostly tons of regular people use it because they're curious to see what it can do. If you make your available market everyone online and then make your product incredibly easy and fun to use, then you're going to get great product market fit. 3. Focusing on need Right now, the era of low interest rates and easy VC money feels like a long time ago. A company that makes hardware based on speculative technology like Leap Motion might not get off the ground. It probably wouldn't raise $100 million. I'm not saying David could see where interest rates were going when he started Midjourney, but he did make a deliberate choice to totally bootstrap the company. David was a known quantity in Silicon Valley. He probably could have gone out and raised lots of money for anything. Instead, he did something way better. He consulted his years of experience and asked himself, what is the scarce resource I need more of to truly scale? The answer wasn't actually money, it was compute. So when Midjourney was just starting out, he leveraged his network. He did stuff like email, cloud vendors and asked to look lock down access to GPUs to scale his service. It's not just about capacity either. David remembered that in the early days of the Discord, some people were getting frustrated with how long it would take to generate high quality images. So today Midjourney actively searches for GPUs in parts of the world where it's night in order to accelerate the process. That dynamic load balancing is what's enabled it to consistently keep rendering times pretty low. Two other entrepreneurs, investors Nat Friedman and Daniel Griffin Gross, have made the same realization as David that there are startups that need compute way more than money. That's why they launched the Andromeda Cluster, a massive cloud server specifically for startups. It's a small but fascinating shift in what's valuable and the value that investors can provide, particularly when money is tight. David's experience shows that you don't necessarily need to wait until interest rates go down again to start a company. You just need to identify what you need most and then focus on obtaining as much of it as you you can. 4. The value of Learning Midjourney gets better by learning from its past experiences. So did David. He spent almost a decade running Leap Motion, and before that he spent years in academia. He's been an active member of the AI community, absorbing the technical knowledge and the philosophical debates. All of those experiences made him a better founder, CEO and engineer. And just as importantly, David isn't done learning. He holds the four hour long sessions with thousands of people on Discord to answer questions and see how everyone's doing. He's constantly tinkering with Midjourney itself and looking for ways to make it a better product. And he's still constantly thinking about the place of AI in our society and how it can contribute to making us better. David gave an interview last August where he really grappled with this idea. He ended up comparing it to water. Water is dangerous, but it's also a driver of civilization. And we're better off as humans who know how to live with and work with water. It's an opportunity. It has no will, it has no spite. And yes, you can drown in it, but that doesn't mean we should ban water. And when you discover a new source of water, it's a really good thing. Lots of people worry that AI will make millions of jobs obsolete or that it'll rise up against humanity. But not enough people think the way David does that AI can be dangerous if it's handled the wrong way, but that it can also be this amazing, life changing resource. It took David Hulz about 20 years to go from starting college to leaving Leap Motion. But it only took him about a year to grow midjourney from an idea into the world's leading AI image generating tool that makes millions of dollars every month. And he didn't need a cent of venture capital to do it. All he really needed was years of experience and the same curiosity he had when he was a kid playing Star Dark Forces on his dad's computer. If you want to learn more about how we should react, regulate AI to avoid making the same mistakes that we did with nuclear, just watch this video next. Thanks a lot.