#33: New AI programming tool

Who are you?

I am Mark Gable, co founder of Solver, which is a new AI development company.

And I'm about to see something mind blowing because I saw it yesterday. Now you know I have a developer tools list of I think 700 something companies on x.com all building developer tools. I'm not a programmer myself, so it's hard to really attack that, that pile of companies. But how do you fit into all that? What are you all doing? Because I assume you're building a developer tools company because I saw it yesterday. So give us a little taste of who is your company. Why does it need to exist in such a world of so many developer tools?

Yeah, really. Actually, as a quick little side note, I'm so happy there are so many AI dev tools right now. This is what I did my PhD in many years ago. And the fact that it has suddenly become the hottest thing on the planet is like a gift to my career. I'm so happy right now. But anyway, Solver. Solver is what we call the first elastic engineer. It's very different from the other AI development products in that you can actually fully delegate tasks to it. You can throw it over the fence, essentially close your laptop while it works, and the work happens asynchronously. And what's kind of very exciting about it being asynchronous is that because it's asynchronous and cloud native and working on tasks entirely on its own, you can scale it as much as you have a need for. So we basically like to say that what we're going to do is kind of provide sort of an AWS console for software engineering. Like when you're in crunch time and you have a bunch of bugs being filed, you can turn a little knob to the right, turn up the Solver effort, and basically have intelligent software engineering ability at your organization or project scaled up like it's a utility.

Can we see it?

We absolutely can.

That'll give us a lot more context for having a conversation.

Yeah. So let me just get my Windows. I like to make sure that you don't get caught in the infinite U right there.

The. The mirror that goes into a lot. A deep hole.

Exactly. And if you'll indulge me for just a couple of minutes, I'm going to do a full screen share and I'm going to show you just a little bit of more of a contrast of demo. So I'm just going to take a minute before I to Solver itself. And I wanted to start off with just a little Bit of context for the viewers here. The app that I have open in the center of the screen right now is called Linear. And Linear is an issue tracker. And it kind of is in the same market as things like Jira or GitHub issues. And it's very common for projects, especially ours, to have a very large backlog of tasks that need to get done. And this is often like where work assignments end up starting with engineering teams. What I have done right here is filed a bug to myself and I'm essentially saying that this engineering manager has assigned this to me. The alignment tests are broken. They're broken on this branch. Try to fix them in the current state of the art. I have a nice full featured IDE right here. This is the JetBrains one. We're 5050 on JetBrains and VS code at our company. We fight with each other over that. And what I've done is I've gone ahead and checked out that branch would have been the first thing I would have done. But from that point forward there's a little bit of like, you know, software engineering needs to get done. So how, for example, do I feel figure out what tests needed to be run? Well, I need to kind of search through the project, maybe use the search feature. I can try running one of them. Look, it has a similar name. Oh, those ones didn't actually fail. That must not be the ones that they were talking about. You know what, maybe it's these ones. And you know what? Oh yeah, there. These are the ones that someone assigned to me. And then once I get into the file, there's a lot of grokking that I need to do. This is a thousand line module. It imports a bunch of very interesting. You must know what these declarations are in order to understand it. So I've got a page into this task I've got to iterate. And one thing I wanted to point out is that we do have GitHub Copilot running here. One deficiency of GitHub Copilot is while it really speeds up your typing when you're generating net new code like this. I can give it that and it'll sort of fill in the body and it's quite good at that kind of thing. It's very cumbersome to use when you're actually editing code. It's hard to get suggestions out of it because I don't really have a way of communicating my intent to it. The only intent that it has is the surrounding code. A lot of times when you're in the middle of code. Like if you're trying to fix it, you'll get these nonsense suggestions that are just saying generate the exact same line of code you just wrote. It just doesn't know what to do. Thank you for indulging me on that. What I want to do right now is just page over to Solver. What I have right here is the current web app that we have around Solver. Very soon it will not be the only way you can interact with it. We have a really robust public API with this where we can basically do integrations with anything. What I'm going to do is start what we call a session on that branch that was assigned to me and I'm just going to give it a natural language instruction right here. The alignment tests are broken on this branch. Try to fix. It's basically just like a breath of fresh air right now, which is. Sorry, I was making sure I could still see you there. Yeah. What we have going on right now is this first step which does take a little bit of time. That thing that said scanning repository, that's getting deep in this entire large repository. This is one of our production repositories. There's a lot of code in it and there's multi year history and it's trying to grok exactly what files it needs to start looking at, to edit, to learn about, and oftentimes to reference or run. So what's going on right here is basically like what I had done in the ide. Those same steps are happening here, but in a very hands off way. And what it's doing is for example, reading all the lines of these files, grokking what the functions are, what could have possibly failed all that thing. Then eventually it's going to start doing certain things like running the tests and actually trying to figure out whether or not they were broken.

This is just amazing. You talk in English to it, you tell it what you want to do, debug this code or do variety of how many different things. It's infinite what you can talk to this thing about, right?

Yeah. And you know, with these agents they get a little bit better when you structure the workflow and kind of give them A, B and C steps. But the less structured that you make their workflow, the more kind of possibilities are there. You know, like if you, if you tell it the way to fix a bug is to open a file, make an edit and close a file, then how does it know how to, for example, update documentation? So we try to leave this sort of open. So this is Actually, Robert, something kind of fun happened right here. So, you know, agents, we like to say at my company, agents do agent things and so sometimes they make mistakes and stuff like that. What actually happened here was it made the mistake that I pantomimed to you right here. Basically that second test that wasn't actually failing, it assumed that that was the one I was talking about and actually went through and saw that it wasn't failing. Even inserted some print statements to actually try to see like, am I missing something right here? And said, sorry, Mark, this wasn't actually failing. So it, it, it took a, a stab at it. And actually one thing that we could do right here to kind of bring it back on the rails, you say. I actually was talking about test alignment. Those are the broken ones. And we can kind of give this as a follow up conversation and try to get it to, you know, make the fix that we wanted. Oh yeah. And so look at this. So we gave it a little bit of a steer right here and said, all right, let's start by running the tests in the test align, blah, blah. Now we have sort of the reproduction that we saw on the ide and now it's kind of going to move forward with fixing it. So I'll just, you know, since we know what this is going to look like, it's going to fix the bug and it's going to do that kind of thing. What I'd like to do is kind of show you a much more involved task and this is the kind of stuff that we're super excited about. So I'm going to start another session on our current main branch, which is our default.

And this is what you mean by Async. You can do multiple things at the same time.

Everything that I just showed you I could have walked away from. And none of it's running locally, so I could have completely closed my laptop.

Too, which is the real power of Agents. Right. You can have your thousand interns all working at the same time.

Yeah, it's, you know that how synchronous Generative AI was until Agents is. There's a lot of buzz right now about Agents essentially being what is finally going to deliver on sort of the productivity value gains that have been promised by Generative AI. So we'll see. We're definitely making some inroads there. So what I'm going to do here is give Solver some instructions to say, you know that code that was just broken on the other branch? Well, on Main, it's not broken. And so what I want to do is give it a very difficult task of getting in there and actually profiling it and making a bunch of performance optimizations to it. And this task is going to take a lot longer to run. It's going to do a lot more intricate stuff. It's probably going to make some mistakes, it's going to try to make optimizations and they're not going to be good and they're going to have to roll them back. So what we're going to do is.

So this behaves like an autonomous car through your code in that, you know, an autonomous car, like if there's a truck in the way of, of you, it figures out a thousand pass around the truck and picks the best one. And that's what it's doing right now. It's, it's doing all sorts of optimizations, testing them, seeing if they're good and if they're not, they roll back and if they are, they keep them.

You know, it's just a really good analogy you drew there too. And you know, it's part of the narrative at Solver is that we actually like to say Solver is the level 4 full self driving of software engineering. And we're still, we, we left one level on the table so we, we four out of five so we still have somewhere to get to. But yeah, that's exactly analogy that we.

Love in your code in a hurricane.

And that first, you know, what we did right here with basically kind of bringing it back on course or reviewing it with one more sentence that's sort of like the equivalent of when you, when you grab that steering wheel in your Tesla. You know, it's like maybe I don't want to be in that lane.

Can you optimize for cost? Can you say hey, I'm using Amazon web Services for my back end. Can you optimize and check if Google Cloud would be better or Azure for Microsoft would be better, particularly for cost, that kind of thing.

This is something that we don't have built in at the exact moment, but I think it's an extremely compelling idea. And really with these agents, as long as you can present a very coherent and somewhat simple view of what you're talking about there to them they can totally make informed choices. And if they can't, you can still do things like we do like fine tune models to actually, we actually have, you know, some of our own foundation models powering this in addition to frontier APIs and stuff like that. And you know, cloud costs are very near and dear to me. They Keep me up at night.

So there's other kinds of costs. Like if I'm building a VR app, I, I don't want a lot of latency, right? So maybe I'll have it chew on, on a certain part of the code and say, can you, can you make it run, run faster or in less memory? Because I'm gonna have to put it on a quest 2 or something like that.

Oh yeah. And stuff like this for like transpiling and moving in between different frameworks or switching. We had tasks where we switched, you know, like what web server was hosting this app and stuff like that. This kind of technology is fantastic for it. So what I've done right here is I've gone back to that first bug which in the end it ended up fixing it kind of like a junior or sort of a mid level engineer would do. And what I've done is I've enabled an experimental configuration and given in a hint and told it to start over. Do it in a different way. And this is something I think is really fun because this works very much like more experienced engineer would work. So if I assign some, say if someone assigns something to me that says the tests are broken on a branch, I'm going to start with the branch, I'm going to look at the history who broke them and when not. Can I just fix them based on the symptoms. Right now what's happening right here is that solver is actually orchestrating a somewhat difficult to use git command called bisect, where it's basically doing this cool binary search over the entire history of that branch, going all the way back to the origin commit and then seeing what commit ended up breaking that branch. What's fun with that is now instead of actually reading through tons of files, it was able to say, you know what, this is almost likely, this is most likely what the bug is. What it's going to do is do a very surgical fix right here. And if you kind of rewind the video and look at the fix that it did in the first version and this version, this one is actually both fix it. This one is better. It does it at the right layer of abstraction. So this whole like we like to use this term, like repository level understanding or repository level reasoning, which is a lot of the value add that we're building into this repository level understanding is about space. Can you fan out and look at thousands and thousands of files and millions of lines of code? But it's also about time. Can you go back and look at the history of a project and learn how things are done so you can do them in that way.

It picks up your coding style.

Oh yeah, definitely. In fact, lots of other really interesting things too, like co changed files. Like if it's the standard in your project that you always increment some particular version number whenever you touch one file, it'll learn that very quickly. We definitely had that at our last company at Viv, where every time we changed something about the platform, there was the secret variable that had to be incremented. Everything would break in production. Exactly. Stuff like that.

Wow. Does it keep track of the people who are checking code in or writing code and does it start rating them as, as their quality? You know, couldn't tell that you're a superstar coder where I'm, I'm not.

You know, we have a lot of that stuff storyboarded out right now. In fact, we, we, we talk about like the single player versus the multiplayer experience with this product and the multiplayer and building in much more sort of team cognizance and stuff like that is not only possible, but super exciting.

Wow. Can I talk to it about like, hey, I'd like to add a feature like I need to make a. Have my users make phone calls. So I need to use a Twilio API call probably. Can I talk to it like that and it'll just build it in?

You absolutely can. In fact, let me show you something right here in this session that we had just finished that had that bug fix in there. I'm going to hit this open pull request button and it's going to actually take this patch that we had just created and it's going to push it to a remote and actually open up a pull request for me. Why did I bring this up as a segue? It seems like a bit of a non sequitur. It's because this feature, as a feature of Solver, we just talked to Solver and told it to add itself. So it figured out how to use the GitHub APIs and everything.

Wow, wow, wow, wow. Can you tell it? I'd like to make this project open source. So you have to make it extra comedy heavy kind of thing.

You can definitely do things like cross cutting, things like that and make it a little, you know, add more verbosity to this. I actually, I think in one of our demo repos for our early users, what we did to make demo repos is we just froze a few open source projects, brought them in, and then for the demo queries that you can click on when you log on we literally just took issues from their backlog that are open right now and one of them was like, prepare this project for the migration to Python 3.12. I said, all right, let's give that a shot, let's see what it does. And it actually goes through and actually figures out the language level things that are wrong right there. Doing the future, removing the feature import where it's not necessary and basically getting it ready for the next platform version.

Give me a sense of other things that this is really good at.

Well, it's definitely good at debugging and fixing things. This performance optimization thing is actually surprisingly good. And it's one of those things with agents, we're all learning what they can do well and what they can't. And this one was a surprise to us with how well it ended up working. We imbued Solver with cognizance of best practices for performance optimization, which is always measure twice, cut once thing. This session that we had started, while we were playing around in the other session, was running asynchronously in the background, all in the cloud. We had asked it to basically make this code faster. The first thing that it did was actually write a profiling script, which is what you need to do. You need to exercise the code, you need to exercise it enough that you actually get a non trivial amount and it's not just a shred of a millisecond. What it didn't then is actually run a profiler and generate a flame graph that shows, okay, here's the fast spots and the slow spots and all that kind of thing. And then it starts iteratively making optimizations while still remeasuring itself and making sure that the test still pass.

So, wow, all in a few seconds.

Yeah, you know, it's kind of engineering.

In the old days would take weeks.

I know this kind of thing would have taken me a lot more like, you know, on the order of hours to at least get started with. And what's really kind of fun with this is, you know, so one of the, one of the sort of limitations of generative AI right now, a lot of it due to like mixture of experts, architectures and stuff like that, is just that it's very difficult to make the output 100% deterministic. But there's a silver lining to that, which is that the variety that you can get out of them can be actually kind of fun with this. Like when I run this demo to try to optimize this code, every time I run it, I see a Different optimization and almost every time it's a good one.

So let's back up. Non deterministic means it's different every time it runs, right?

Yeah.

Adds a little variability into your code base, which is a little scary for some programmers, right?

Yeah. And if you were to do sort of like a stock agent, like if you were to kind of like build a hackathon right now and say, all right, let's, let's crank out a software engineering agent in two days, that version would be a lot more non deterministic than Solver. We've actually done absolutely everything in our power to try to make same input, same output, but really there's just certain uncontrollable little bits in there. But yeah, basically on a scale of 1 to 10, we're more like 7 out of 10 deterministic, but still you see variance in there.

Can I tell it to build a different code base for like, if I'm building an app for a Vision Pro, I'm probably going to have different API calls, different architecture, different approach than if I'm building for the Quest 3. Right. Just to use a VR headset metaphor, can it do that? Can I sort of say, okay, we're going to split the code base up into different platforms, Macintosh versus Windows or kind of thing.

There's definitely system level tests you could do as long as you give it the right context and kind of give you give it an idea of what you want to do. You'd be amazed at some of this stuff. Our code base is obviously polyglot as any significant one in. So we have Ruby code that's called from Python code because we can only get into the right library via Ruby and this and that. Now we're integrating a Rust library for doing some more of this compiler front end stuff.

Oh, that's an interesting point. If some programmer likes to write in Rust and I like to write in C or something, does it convert their code to C so that it's all in one language or does that even really matter anymore?

You know, Solver is not going to without instruction voluntarily just change a bunch of code in your code base like that. If you have an existing code base where you have some Rust code and some C code, it's going to do its best to work in concert. If there's Rust bindings for the C, it will know that if you work on the C stuff, you also have to update the Rust side and things like that. If you want to see if it can translate the Rust into C, you gotta ask okay.

And does it do it? Does it translate all sorts of things into other things? It's like.

It definitely does. I, I'm not going to be able to sit here with a straight face and tell you that it's able to do that 100 quality 100 of the time. But it's amazing the stuff that it can do.

Wow. And you know, generative AI does hallucinate every once in a while make a mistake. How do you guys. How does that have. How do tell me how you make sure that doesn't happen very often or.

Oh yeah, that's something that's very special about agents. So unlike a chat GPT or a GitHub co pilot that sort of lives within the IDE, it doesn't just generate code. And the hallucination stuff with that kind of thing is, can be very frustrating when you get into low resource languages or low resource libraries and frameworks. By low resource I mean there's just not a lot of data. So we know that GPT4 wasn't exposed to very much of it. So for example, like when I've been working in Pygit 2 to do it's a library that lets me in a very low level interact with Git repositories. When I'm using GitHub Copilot, it will generate code that looks absolutely perfect. I'm like, oh man, this is exactly what I needed. This just saved me so much time. The API it was using did not exist. It was hallucinated and it was basically something that probably should exist, but just it just said, not that this is probably, probably what you should call if it existed. What's fun about agents is that there's so much more kind of like backstops things that are keeping guardrails that are keeping them in sync right there. So I was scrolling through that session really quickly when it was doing performance optimizations. I scrolled by kind of quickly. I'm not trying to dwell on that kind of stuff right now, but I spotted with my eye a couple of times in that session where it made optimizations that didn't actually work. So they actually, you know, they might have made it faster, but you could see the tests failing, saying, oh well, yeah, you know, with the best optimization I can make any program run in a 0 second I just delete it.

Wow.

Yeah, so that's the kind of thing where when it's free to execute code, when it's free to run linters and check for security vulnerabilities and basically do the same type of Things that we do as software engineers with the same tools that we have. There's so much more that's keeping it on the rails and reigning in the hallucinations.

That's cool. You probably have deep thinking on how this is going to change programmers, how AI in general is going to change programmers lives, but certainly your tool, how is it do programmers have a job in a decade?

I firmly believe that they do. And they're not only going to have a job, but they're going to have a more fun and better job. I mean this very sincerely. I have a much younger brother graduating from UC Davis with a CS degree and he's on the market right now. I am very, very bullish on the CS job market going forward. What I see this kind of technology is not really displacing workers. So right now demand for software is effectively infinite. Like basically as humanity, there's orders of magnitude more software that we wish we could write that we just literally can't right now because we don't have the resources. And it's like a known and studied fact that like a good 75, 80%, whatever number you want to call it, somewhere in that kind of frame, you know, that sort of ballpark is basically all scut work with software. Like at my company we always love weeks when we can make a new feature or hit a new benchmark high on something like that. But you know what, we have a lot of weeks where it was, we have our end of the week, drink a beer, have the, you know, the team meeting and all we ended up accomplishing was fighting with insidious infrastructure issues where we ended up having some little race condition that consumed the whole team's time for five days. If I could sort of sweep those weeks to the side and leave my team free to spend their mental capacity on building the right thing instead of just iterating on the one thing that we're building. Oh man, amazing things could happen.

How does the developer team need to change as they get into using AI tools? I mean some are already deep diving in and going to San Francisco hackathons, AI hackathons and stuff like that. But others are like still using Visual Basic code. How do you get the. What does the old programmer need to change to get into this new world? Non deterministic thinking is one of them, right? This idea that the LLM can give you a different answer every time you ask is sort of weird to people, right?

Yeah, it is. There's a. When we were raising funds for this company, one of the analogies that I drew to potential investors was essentially like, you know, the early 2000s IDEs were the new thing. And if you were the type that, you know, still goes completely manual, uses VIM without any plugins and stuff like that, which there are still plenty of programmers like that today and I'm not going to, you know, talk down to them, but you know, ides were a sea change. It was one of those things that when people started using them, the predominant opinion was oh my gosh, this is so bloated. No one is ever going to use this in the long term. I'm going to stay on what I'm. And then pretty soon within years it kind of flipped because people just, it got out there that this is a.

Lot faster to drag a tool out of a Visual Studio toolbar, drag it on a form and make your. Make your ui. Right?

So I mean you're exactly honing in on the point that I'm kind of making is, you know, for those that are kind of trying to avoid this, you can avoid it for a while, but this is going to be one of those things where at a certain point you just can't afford not to learn about it. So the fun thing is you said you had a list of like 700 plus products that include mine. My best advice is to just try them because Generative AI has a little bit of like a. What we describe at our company, a little bit of like a tactile feel. You get a feel for the kinds of stuff it can do well, the non determinism issues, these things sort of cross cut across all the products and trying them really just gives you an idea of what this can do right now.

Yeah. What other kinds of tasks? Setting up unit tests. So you covered that a little bit. It does that real well. Right.

Unit tests is generating new tests. Generative AI that's been a very, a big, big win because testing code tends to be very idiomatic and idiomatic stuff structures very nicely into a language model. So it's really nice for that. We find it very useful for maintaining and cleaning up non code artifacts too, like a lot of configuration. So we have Docker files that we have to keep in sync with JSON configuration files. And these things have to be done in lockstep and we need to make sure that when this Python requirements file is updated that the one for the docker image is also or else it crashes in production. Because these types of like not quite DevOps but a little bit closer to that than programming. We find it to be very useful too.

Wow. What other kinds of tasks do programmers want to use this kind of tool, I mean, this kind of system for? Does it, does it rate your architecture and say your architecture sort of whack here. You know, maybe you should think about, you know, putting this over in this component or this. Right.

If you ask it strongly, you can get it to do little evaluations and stuff like that, but we definitely steer and train it away from first and foremost doing arbitrary things that you didn't ask for. So steerability is very important. And second, from being too opinionated.

Does it have a personality?

Depends exactly on the configuration too. We have like a little bit more of like your IBM kind of boring configuration. One where it's a little bit more. A little bit more loose. But yeah, yeah, yeah. Other things is, does this mean you.

Need to buy more screens because you're running four different things at the same time, you know, and then telling it to do different things on the different windows? Does it work better if you have a three or four screen?

I think about like day traders right now, but yeah, you know, if you are.

Microsoft actually did research to find that people, programmers at Microsoft are more productive with two screens back in the old days. Right now I would say it's my. Well, get a Vision Pro, you can have as many screens as you want.

Right. That thing is amazing. Yeah. Like I mentioned before, we're trying to build this a little bit more like a scalable utility where things are really running in the background. And so the idea that you would actually have to watch them like we were watching the demo. It's not what we're building around, but I have to admit it's really fun just to watch it.

Yeah.

So when we are experimenting and working with stuff, we are totally using multiple screens and multiple windows and watching to do this. This Sometimes we're running the exact same thing because the solver is all cloud native and it's just we can do it on a fresh branch. Sometimes we're running three of the same tasks with slightly different instructions just to kind of see what interesting things it'll do. It's just fun to use synchronously. But like I said, the vision of the company Async, it's a utility. Engineering. Turn to the right. Lots more engineering.

Yeah. Can you have it do a security audit on your code and find, hey, you got a backdoor in your code. Somebody put in or here's a condition, some bullshit. It'll go into a weird condition and let a hacker into your System, right?

Yes, certainly, like without any, you know, I'll add a little bit of a little disclaimer there because of course, you know, like program analysis is a very hard problem. With Rice's Theorem, you either get soundness or completeness, but not both and all that kind of stuff. So Solver is not going to prove the absence of vulnerabilities in your code. But we actually have built into the product today the ability that you can pull in an Internet enabled security linter that can look up usual CVEs and stuff like that and actually scan code bases. And the fun thing about it is normally a lot of those Linter errors, they kind of just get mothballed because engineers have more important things to do. But for actually running through the code and systematically proposing a fix to each one, it works fantastic for that. I think we actually have a video of that. I'm not sure.

What questions am I not asking because I'm not a computer science major and been writing code for 10 years. What kinds of things would those people be asking you?

Well, I think the AI savvy ones are probably going to want to know a little bit more about what's under the hood.

Yeah, please let me know. Give me the secret sauce can tell me about your secret recipe, your Coca Cola recipe underneath.

Yeah. So you know, agents, it's not a single model thing usually. So I was frank earlier when I said that we're using all the best of the frontier stuff right now. So everything that you saw today was powered, you know, a little bit by GPT4, a little bit by Sonnet and a little bit by gpt01. It's very nice to use in certain surgical cases. But it was also all kind of tied together that whole knowing how to navigate the code base. We have our own proprietary model family, the largest version of which we call Total Hunk. We had the little ones who are called Mini Hunk and stuff like that. What Total Hunk is, we haven't named that funny name because it's largely built from the raw material from git commits and diffs and the related artifacts and issues and stuff like that too. We basically did a, an Internet scale data set exercise where we ended up building over 5 trillion deduped and clean token data set of 20 billion git commits. And that's actually what created Total Hunk. So Total Hunk isn't an absolutely massive model, but it has seen a massive amount of data and it's seen much more importantly than most coding models, projects at various stages in their development. Not just a snapshot at the end.

Eric Reese said he once coded for six months on a thing and then put a link on the website and nobody clicked the link, nobody downloaded the code. In other words, he didn't do the market research to figure out if what his team and him was building was going to be useful to anybody. Does this do any marketing analysis? Like oh, you're writing some code, maybe you should think about doing something else with your.

Yeah, I think you've got me there. But one thing that you just kind of reminded me of is, you know, you asked me that nice question earlier about what does this mean for engineers and their actual jobs and stuff like that. And you know, one thing that has sort of pervaded my entire career is busyness. We never run out of stuff to do. You saw the backlog at my little company and while we were for example Viv, that was acquired by Samsung, we had a really really great experience there. But Samsung stuff is really, really, really, really busy there. There's, there's a quarter million employees at that company. You don't realize how big it is until you're in the middle of it. And during one of the years we kept so busy and then we found out basically at the end of the year that our bonuses collectively were all being docked a little bit. And so what causes that? We all did our work, we put in effort and everything. But the thing is that the real world doesn't reward effort. The real world rewards results. So the idea that maybe during that 12 months the thing we were iterating on maybe wasn't the thing that we were supposed to be building. That's the type of free and abstract thinking that I want to empower all engineers to be able to do. Instead of being bogged down in this day to day, try to close as many tickets as possible before lunch.

Wow. If you are one of those people closing tickets before lunch, it keeps track of that. Like oh, you know, Mark's a good programmer, he closed 20 tickets.

Oh yeah, that stuff happening even without AI, there's a lot of orgs that use metrics on issue trackers and stuff which. Which ends up to very perverse incentive. Sometimes, sometimes you break up the tickets into multiple tickets so it looks like you solved more of them and things like that.

But yeah, are you finding engineering teams are use. I mean there's 700 companies who are building developer tools. What's their developer stack looking like? And how does this fit in with other AI tools that they might be or other developer tools that they might be using.

I am so glad you asked that question. One thing that is a just a common theme among literally everyone who's using Solver right now is that everyone loves our web app. It's fun, it's futuristic. Everyone asks for a way to not use our web app. Yeah, so what we really, you know, super high priority for us right now is meeting developers where they are with their existing workflows. We have developers that want to launch Solver sessions from Slack. They have a chat with their partner and then they just say slash solve in this. And then we have developers who want it to just be able to chat with it on GitHub with messages and we have other ones that want it to be part of their CI system. Well, we're going to enable all that and actually very soon the highest priority at our company right now besides this go to market that's happening is behind Solver and its web app that you saw is a very robust platform and an API that is actually meant for public consumption. We're actually going to not only have an API that's going to enable a lot of very easy integrations, all the IDEs, all the workflows and stuff like that, we're actually going to start selling access to the API. But I believe for the moment we're going to be the first to actually sell access to an API. For a software engineering agent. There's a lot of really tricky designs that goes in that too. What happens with an API where you, an API call might take 20 minutes or 5 minutes or something like that. There's a whole design and kind of life cycle that needs to go around that.

The ultimate latency problems.

The answer is always Async. Async. Async. Async.

Async is the pro, is the answer. Huh?

Yeah. Set it and forget it.

What else do we need to know about the world of development? You know, how does this fit into Agile workflows? Right, Because I, I keep at Microsoft I saw the beginnings of that kind of like flip over from methodology of waterfall where one person writes all the code and then hands it over to another person who writes some more code. Tell us a little bit about how this changes the methodology of a development team and maybe how they would work together with each other.

Yeah, well, one thing that comes to mind, which I don't think is going to be a total answer, is that technology like this and a lot of other AI driven tools enables faster and higher fidelity prototyping. So I can give you an example, like literally just yesterday we were talking about something that we didn't particularly like about Solver, which is that sometimes you saw in that demo it looked like a wall of text. There was something missing in between those walls of test that maybe gave it a little bit more context. And we kind of had a little bit of a debate about it internally after I finished talking to you yesterday. And instead of just debating it, someone on my team just use Solver to implement a quick and dirty version of it. And then we actually had something super high fidelity that we could see with our eyes and say, look, we actually do like this idea. This idea was good. And instead of having a meeting about it and debating and drawing what it looked like on a whiteboard, we literally got to see what it looked like in the product. And actually the code's not half bad. We could probably just land it. So that kind of thing, to be able to. The freedom to try multiple more ideas, I think is going to be really, really good.

Tell us a little bit about the company you're building. I mean, you came out of the Siri and the VIV team. Tell us about some of the expertise that's going on behind the scenes here.

Oh yeah, which I told. This is the, the, the best team I've ever worked with. I'm super proud to be working with everybody. Like you said, there's a lot of history that comes with the Siri and the VIV people. This is a lot more of like the platform and ML people from that group. For kind of obvious reasons, we have really, really amazing levels of generalism across the stack. Like my front end developer actually helped clean data sets at certain points and stuff like that. And like I mentioned, we've trained foundation models, we've had cluster issues trying to figure out what, 256A1 hundreds, why are they not communicating with each other properly and getting a nickel error on this kind of thing. So we've built a lot of really great competency through that as well, which we did a lot of that stuff at Samsung as well. And you know, the other thing I like to say about this company is that it is like kind of my life's work. So the fact that, you know, I did my doctorate in this kind of stuff, I actually was part of a group of academics that was the first to put a bunch of code through a language model, which I think in hindsight is a very obvious idea. But we did it, you know, a long time ago. And then I spent 10 years of my career building real AI products and shipping them to just tons and tons of people and learning how super important the product side is. And this company feels like it's bringing both of those threads together and it's just extremely gratifying to me.

It's a good place to end. Where do we learn more about it and about you?

Well, the website that as of this moment doesn't exist but by the time someone sees this will is going to be solver AI.com Very cool.

Thank you so much for showing me this and showing me how, how development is changing. It's amazing. You know, it's a new world we're in.

Thank you so much for your, your patience and all that really, those really, really thoughtful questions.

Love appreciate what you do and you know, because a lot this changes the world and this is not very many people have recognized that unless they're in the computer science world. But what you've been doing for the last few decades has already changed the world. But this is another one. So thank you very much.

Thank you.