Jerry Chen and Instabase CEO Anant Bhardwaj | Building a System of Intelligence

Foreign.

Hi, welcome to Grey Matter, the podcast from Greylock. I'm Jerry Chen, general partner. Today we have Anant Bhardwad, CEO of Instabase. Instabase is a great case study, what I call the system intelligence. It's basically building a deep mode of IP around AI for the enterprise. And so today I want to bring in Anantbard Wallah to share his story about AI Instabase and use this as a forum for us to talk about lessons around the new modes. Anant, welcome to Gray Matter.

Thank you Jerry for inviting. I'm super glad to be here.

Anand. Instabase is the applied AI platform for the enterprise. And in my mind this company at the most basic level allows enterprise to build apps using applied AI to solve random business processes. Right. Software is always a business process like order to cash hired a fire. But whereas those processes were turned to apps a long time ago, most of the work inside the enterprise has not been digitized. And then along came Instabase. Right. So you basically built this from the seed funded company in 2015 and you just raised at a $2 billion valuation. Congratulations on that. Why do you think the market's so excited about Instabase? What is it driving the demand and the interest from both customers like the top banks and investors alike?

This AI is evolving very, very quickly. And I think where people are looking at is where is the most value creation that could happen. So we know certain area like number one is you need GPUs to run them. That's why you see Nvidia stock going up.

That's a stock we all should have bought.

Yeah, so that's the first value creation. The second layer is all the cloud providers because they need to run on that infrastructure. Right. So that's why you see all the cloud stock 30% up from beginning of the year. The third layer is the LLMs itself. You need one of these large language models that are capable to do that. And the fourth layer, and that is an open field today, is which platform is going to enable, because AI is going to be indispensable part of any technology solutions that would be built for the next coming years and decades. So which platform would allow these large organizations, enterprises, individuals to go and build those applications, will accrue that value.

Yeah, that goes to this blog. Then obviously. Thesis, you and I have talked about the new Moat originally six years ago, which kind of shortly after the instant based investment and the new New Moats update that you read and help review around, where is the value accruing for sure there's value accruing to foundation models, to the GPUs. And then I think potentially I talk about the system of intelligence, the system of engagement.

Right.

So clearly I look at Instabase and you guys have built maybe the definitive system intelligence company out there for enterprise AI. What gets me excited is that could be an area that does accrue value. I'd love to understand why you think this area that Instabates plays in is the right market for startups to enter, as opposed to build another foundation model or build a cloud hosting provider to run AI models.

And those are all right kind of technologies to build. I'm not saying people shouldn't build it, because more foundation models, more competition there, foundation models will get better. That's great. But I think there would eventually be only some four or five foundation models that would end up winning. And the reason why is all of these foundation models have their own quirks, they have their own way of how do you do the prompting, how do you optimize things? And when you look at sort of like the layer that is above this, it's very, very hard to support more than four to five underlying platforms. Right.

So tell me about where this idea for Instabase came from. Maybe start at the beginning, MIT. You were a grad student building something called DataHub. So tell me about Data Hub and tell me how that was the inception moment for Instabase.

Yeah. So this idea of Data Hub was how do you make application run on something that would not. That needs to scale beyond a single machine? Because anybody knows how to build an application that will run on one machine. But how do you do it when you cannot, your data is large enough or the complexity is large enough that you cannot run on single machine? So the whole idea was like, how do you create some distributed operating system kind of platform where you write the sort of application and the operating system takes care of scaling it, making sure that, you know, application can take advantage of multiple machines and memory and CPU and all those kind of stuff. So that way, the whole idea of Data Hub, where you have an operating system under the hood, you mount all your data files, whatever those things are, and then build application on top of that operating system and the OS will take care of scaling, fault tolerance, portability, all of the standard stuff that otherwise you have to deal with.

There's so much data in the enterprise. What Data Hub did is like bring all the technology which evolved into this applied AI platform to the data. And this was what, 2013, 2014, when you were working under a thesis?

Yeah. So this was a project that started around 2013 when the big data used to be the thing and all the investors, you know, you guys were in cloud era, I think, so everybody was investing in big data where the core idea was like bring data to the compute, right? You basically start those data, move things around. And at mit, the philosophy was, why would you bring bigger thing to smaller thing? Rather you bring the smaller thing, like your code and other thing to the data. That's where it started where you kind of like abstract all of those data pieces on these abstract the layer called Data Hub and people build application on top and then OS will take care of breaking those application into different parts that can be sent to where the close to data it is and then give the response.

So then in 2015 you left your PhD program at MIT and and you took your work in Thesis and Data Hub and you started Instabase. Tell me a little bit, just why did you do that? What dream of dropping out of a PCD program, disappointing parents across the planet to start a startup. What motivated you?

That? It's a funny story. So basically at mit, you have a bunch of investors come and see cool stuff. So I will not take the name of the investor, but some investors came and said, this looks super cool, you should drop out and this could be a company. And I was naive enough, I was like, great, I did drop out, came here and talked to that investor and he says, I have to talk to my partner before I can give you money. Like, oh man, that's a bad idea. And they did not fund, actually, they.

So the investor that gave you the idea to drop out didn't end up investing.

Did not end up investing. So I asked Mate, who is a very good friend of mine, professor at MIT during that time, and also co founder of Databricks, he basically made intro to Pete Sonsini and Peter Levine from Andreessen. And then of course I met Pete and Pete made introduction to you and that's how this seed round happened.

Greylock was very happy to be part of the seed round back in 2015. And so that was almost eight years ago, right? Seven and a half years and counting. And the product has evolved a lot. And so let's talk first, what is the problem instabase is going after? Like, what's the exact problem we're solving?

Yeah. So the core idea for us was like we were building a platform that would help large enterprises automate some of their most complex business processes. Four of the top five US banks are our customers.

Okay?

20 of the top 30 banks globally are our customers.

Okay?

Some of the names that we can say is, for example, Standard Chartered, AXA, NatWest, US patent and trademark Office, Colgate and so on. So let's take an example for a large bank, what are their business processes? What they do? They onboard customers, they give you loans. So let's take one of these examples. Giving loans. So when they give loans, they have a website where people apply for loans. How do you apply for that? You submit bunch of stuff like this is my pay stuff, this is my bank statement, this is my property paper, this is my other stuff. Based on that, they will do the analysis to see that whether you qualify for things and then they will give you a loan. This is all manual, right? And how do you automate that? There is not one application needed to do that. It's a complex workflow. Like first you need to be able to read any kind of these images, be able to OCR it, then be able to classify what kind of document it is, then be able to extract all the relevant information. Once you have that, then apply business logic. How do you put all of those things together into a single application so that somebody applies for a loan? In less than 5 second you get an answer, and that's what Instabase helps you do. And now you can kind of like replicate this to any industry, like for insurance, for government, for healthcare, for cpg, because they all have these similar kind of business processes.

So it's any enterprise that has a complex business process that in the past couldn't digitize or couldn't automate, you bring this applied AI platform to solve the problem for them. It's what we call multimodal now, right? It's paper, it's data, it's APIs, it's multimodal of all the inputs. But then what you're really doing the magic is using AI to kind of automate this process.

That's correct. So earlier, I think some technologies exist where if you could write some simple rules, so RPA you might have heard of, which is called Robotic Process Automation. You can define some set of fixed things and if you do exactly the same way, but the issue is there is no intelligence in it.

And so Instabase just updated your product with something called AI Hub, I believe, and it's built on top of foundation models, correct? That's correct, yeah. So let's go back to this early days of instabase. So you just raised a seed round from Greylock, another vc, and we Actually hosted an incubator in our office in San Francisco, those days when we're in the office. And he built the first version of what was Instabase. Who were the first users? Tell me about that. And how did you go from there into selling to insurance companies and consumer product companies?

Yeah, we literally had no idea who would be the customer. So we basically, you know, one thing, academics the students. So at mit, Data Hub was being used widely by a bunch of data scientist kind of students, PhD students, who were doing some analysis. So we were like, let's build notebook, because that's what data scientists use for doing analysis. And we built that product very, very quickly, I think within three months of the seed round. And then we gave to a bunch of professors. So at Columbia, at Stanford, at mit, at Chicago, and professors will teach classes for them. It was a free compute. Right? Because let's say if you want to give a homework or assignment to students on, do this analysis on entire Wikipedia datasets. I mean, like, basically no students can download all of that on the laptop and run things. Now you put that on Instabase, fire a notebook, start doing analysis, and the OS will make sure that we can run that scale. So that was basically our first set of users. Then Dean of Engineering, Stanford Engineering, Jennifer Widom. So she had started teaching all of her classes worldwide on a world tour. So she was like, can I use this for my world tour? Like, sure. So we sponsored that event and this was being used worldwide, which was great. But then I realized. And we realized after like about a year and a half or so, nobody paid money. So students love using free product. But, you know, how do you make money?

You just said as a student, you paid all your money in rent. And so free food and free office space was nice. Yes, the students are on the same boat.

Yeah. So we were not making any money. So what we would do is one of the things that was always true in Instabase. Whoever came to meet us, we forced them to see our demo. So we'll just do demo to everybody, doesn't matter who you are. So one time there was this company called Genefits, and somebody from Genefits was visiting and we were showing this school demo of Notebook and we had built another app at the same time. It's called Refiner, where you can take log files and extract some value like IP address and event and those kind of stuff. And he was like, this is cool. But, you know, our biggest problem is we allow people to select the best health insurance program on our HR portal and all of these, like health insurer give us this like PDF with the plan. So can you basically ingest that into our database so that people can select the best plan? We're like, sure, we don't have PDF reader and all that, but we had something and we gave and like, oh, this is super, super cool. Jennifer went through some problems and a lot of leadership changes happened so we never closed a deal, but we had a very simple PDF reader. Now we will show that demo to everybody. So David Lunker, our first employee was in Tahoe and was showing a demo and a friend, a friend from Lending Club was there and he was like, we also deal with PDF, not the PDF with text though. People take a scanned copy of pay stubs and bank statements and tax documents to apply for loan. Can you do that? I'm like, we don't do OCR and all that, but we just hacked together some open source stuff because we were operating system so we could put more apps that we put those apps and connected together those apps using our workflow system. Those were the first customer to actually, you know, try in production. But the friend was not powerful enough to close a deal.

But I think those early customer requirements led you to build what became a platform. Right? And so it was like each time you had a new need from a different customer, you said, oh, I'm going to build this platform. And it became super flexible. This system intelligence for AI apps.

Exactly. So Data Hub already had ability to kind of like connect to any data source because we had the file system support, database support. So we already had how to handle any kind of data. We didn't know what applications and over the period we kind of just talked to a bunch of people and eventually I think we got pulled towards this being able to solve these complex business workflows where you get some kind of unstructured information and be able to make a decision.

So let's talk about that. How do these customers use Instabase and solve these workflows?

Yeah, so Instabase started with kind of like low code pro code which is sort of self contradictory thing. So it's easy enough to get started like Excel, right? You can go put stuff and start writing simple formula, but if you want to write very complex code, you can actually go into the scripting and write stuff too. So our flow is built on very simple like drag and drop interface, like process files define different classes. Like this is a pasta bank statement, whatever for from extract these kind of data to the.

So I have a drag and drop user experience to say, read a document, pull this from a database, do this next step B, step C. But if I wanted to, I could pop the hood and go deep and adjust things and code myself.

Exactly. So we basically had something called UDF user defined function. And there you can write whatever you want. Like operating system, we can run any piece of code in a secure way. So that UDF was one of the most powerful feature, because any feature we did not have, we can code it for them. And that allowed us to really understand the market and need. And now if you see that that piece of code is useful across like 100 different use cases, convert that into drag and drop. User element. Right. UI element.

And so that's a lot of the genius behind Instabase to solve all these problems. Workflows. Because the problem with workflows historically was they're so broad, such a long fat tail. No one application can handle it, but Instabious was able to handle it because your ability to do kind of these custom workflows and UDFs.

So in a way, yes. But also it has to be powered by a lot of intelligence under the hood. Because the point is UX can only solve some problem. Right? But under the hood, like for example, if you have to write a rule for every kind of page tab, how many different formats of page dev exist? So you can't do that. Right. Like, say, would you write code for every single type? No. Only technique that works is artificial intelligence or deep learning.

I'd be curious to talk about, like underneath the hood you're using. Is it OpenAI? Is it some of the open source models? Have you looked at all of them? And I'd be curious, you said they all have their own quirks. As a user and practitioner, how do you think about this layer, the foundation models?

So this is evolving very quickly. So anything that I say today might be irrelevant tomorrow. So I think when it started, OpenAI, GPT 3.5 and GPT 4 and GPT 3, DaVinci, all three were state of the art models.

Got it.

And I think they continue to be some of the best models. At the same time, I think now you have more options. You have Google, Palm, you have Anthropic Cloud A, and then Facebook just launched open source. Llama 2.

Llama 2.

Yeah. And some of these models are very, very close to OpenAI. GPT3 and GPT 3.5. I think GPT4 is still ahead slightly, but in general, I think that this competition is heating up and it's very Very hard to call out which one will have the best result.

It's hard to call out. And I appreciate the humility. Say what you say today may be irrelevant tomorrow. Everything I say today is irrelevant today. So at least you have that on me. Why do you think there's only going to be four or five foundation models out there?

So look at the history, right? Whichever had become the defining platform on which applications are built. So at the very, very lowest layer is your CPU architecture. We ended up basically all going with x86, x64. There were only two or three architecture on which most of stuff got built. AMD and Intel. Those two architecture became very common after that operating systems there were Windows, Unix and Mac. Those are the three common operating system architecture. They're variant of Unix, but they all follow pretty much similar principles. Then if you look at cloud providers, they're like you can say five big players. Beyond that I think the market is very, very, very small and fragmented. If you look at phones, there are two big players, Apple and Android. So I think at the large language models which become such a databases, for example, is another example like Oracle and Mo, Microsoft SQL Server, there are like four or five big players and then you have this long tail. And I think the same thing would happen here where you will have like three or four big players who will capture like 80% of the market and then you have long tail, a bunch of small models.

Well, I would think for the CPU architectures, operating systems and to some set cloud providers, what's driving the consolidation are you is network effects, right? In terms it's hard to write low level operating systems to multiple architectures, it's hard to write applications for multiple operating systems systems. And so there's a network effect with the number of developers. So are you implying you think there's also network effects for foundation models?

So not just network effect, but also.

The.

Sort of like the economy of business in the sense that like if you cannot capture enough market, it might not make sense for those LLMs to maintain and update. And, and what basically it means is that there are only four or five big ones that can update at the pace at which they can serve the market and they become better and better and then you have the long tail that would not be able to compete because they cannot invest because they don't have enough customers or people paying.

You're talking our love language of moats and defensibilities. So network effects is clearly one. And your argument is these large models are so expensive to build and train Et cetera, that to your point, economies of scale or business, there's only a handful of companies that really it only makes sense for only three or four companies to spend the billions and billions and billions of dollars to train these large models. Inference is a whole different story. But at least building these models, only three or four companies makes sense because you have to recoup that investment over time. Right. If it's going to be 100, 200 billion to build and train, it doesn't make sense for the 21st model because the first two or three are good enough.

Exactly. And the point is that they every single day new information gets created. How do you keep the model up to date? How do you keep model relevant with the new information that is getting produced every day? It's not just about creating and training the model one time. It's about basically keeping the model state of the art up to date. And that is an expensive endeavor.

There's not a debate. But there's a question of if you think about large versus small models, is large models the right answer and they'll solve all the problems out there or do you think you have a bunch of these smaller models that are built for dedicated problems?

So I think this comparison that has sort of come now is kind of like bunch of these small models already existed. We have lived in that world and we know like they solve some problems. The reason why everybody very excited is what these large language models can do, right? And I believe that over the period, these large language models would really be so powerful that they should be able to do most of the things and most of the things.

So even these small models, from like creating my AI avatar to like, you know, auto correcting my spelling, these large models can do. They can. But aren't they but laments, aren't they overkill? Like why bring this huge model GPT8 or Palm 20 or Llama 14 down the line to solve autocorrect of my typos. And that said, ducking on my text messages still isn't fixed yet.

But yeah, yeah. So I don't think they're overkill if you can run them at scale, right? So basically, point is you don't run that model only for a specific problem. You run that model for all the problems. Collectively, they're able to handle a lot more problems so that they are even more economical to run than if you had to like run 20,000 separate models.

And this is inferent because it costs a lot to train and build. But to run it at scale and Inference and execute.

You're saying the idea is that today the architecturally like in order to do any inference you still have to activate all the neurons in the sense that they are still a bit expensive. But I think there is research that is being done on something called sparse activation.

Okay.

That means you don't have to do like, like the sort of entire neural net to be activated to be able to. Because there might be only some subset of neurons which are responsible for giving you the answer. And can you find that subset in a way that you are only doing smaller number of vector computation? That means you can run it in a very, very, very cheap way.

Let me talk about this is kind of the brain analogy, right? It's like we have all these neurons in our brain and they work together in concert to solve a problem, but you don't use all the neurons at the same time. So a, the neurons all need to be cohesive in the brain for it to function. But if we can identify which neurons to fire in this AI brain then you can actually do a more cost effective. But that's not a solved problem yet, nor is it solved problem in the human brain either. But it's definitely not a solved problem in the AI brain.

There is enough progress being made that I would see that that problem would get solved. And if that problem gets solved then running large neural network like of course you have the initial cost, but the amortized cost is going to be much more economical than running large number of disparate models.

Because originally you weren't using lang's models in 2015, 2016. So you were built on smaller models beforehand. And then what happened in 2017? What were these small language models?

So until 2017 we used two different techniques. One was program synthesis, kind of like give an example and can machine come up with the code itself. And the second one is bunch of smaller logistic regression and knife based kind of models. They work for simple problems, but they don't work for large scale language understanding and cannot handle the variability that the new generation models are able.

So in 2017 the transformer paper came out. What happened then?

So we basically looked at a paper and that paper is an amazing, it's basically very, very powerful paper for language understanding. But the issue was they encode language as sequence of text and you produce sequence of text. And that works for like spoken language or web language. It does not work for document because for page stub it's like it's a layout that matters, right? If I say net pay year to date, you have to look at that column and look at that row. And there was no way to encode the layout information. So we basically wrote our own language model called Layout Aware language model where you start encoding X and Y coordinates and those things. Rafael who leads our deep learning team, he wrote a paper called Text Image Layout Transformer and then Layout Aware modeling of Bert called Lambert. And that's what led to Insta lm. This was our first language model that came out in 2020. So we were in this language model game much earlier, before it became super cool.

Well no, you took the insight that they had from the transformer model, predicting the next word coming in the linear path and so saying, okay, how do we take this 2D representation and apply that same logic to a layout on a piece of paper that has X and Y coordinates to predict? Okay, instead of predicting the next letter or word in a sentence, let me predict in the next cell going down or going across, et cetera, anywhere. What should this be, address is paste up or whatever. So this Layout Aware models like instalm, external, et cetera, you built through 2018, 2019, 2020, was that powering the base at the same time? Do you still use that? How do you use that in coordination with the large language models today?

Oh, I mean 90% of our revenue until like launch of AI hub is still through Instylm. And Instylm is a very, very powerful model. The only requirement in that case is you have to fine tune this on customers data. So for example, if you have to make it work on let's say Greylock annual statement, then you have to take 100 Graylock annual statement, sort of like annotate the things that you want and basically say fine tune this model and it pops out a new model called Greylock animal statement model.

But we're seeing this today, right? So a lot of people talk about fine tuning models for XYZ company, ABC organization. So fine tuning is a common approach to solve those problems. But there's some drawbacks, right?

So with large language models we believe, and the bet that we have taken is that you would not need to do fine tuning. So let's understand what fine tuning is. So what fine tuning does is each. So these models basically are just a. It's basically a matrices of weights, it's a bunch of floating point numbers. So when you fine tune, these numbers change. So some number from 0.8345 will change to 0.3456 and you have no idea what they really mean, but they change in a way that it becomes very, very good for the data set on which you fine tuned. But you have no idea what effect it would have had on any of the general purpose problems on which it was trained earlier. But what we have found is with large language models without fine tuning you can get comparable accuracy, comparable means. Like if the fine tuned model can give you like 97, 98, this can give you 96, 97, so pretty close. And now this single model can do any use case, so you don't have to worry about collecting data.

There's a cost to train and fine tune, etc.

But also no access to data. So I can only give it to you.

Yeah.

And then you have to train it. So extra effort for you. Like it takes some number of weeks to get value here. From day one zero sort things work. And I think that that will become more of how people would do things than the fine tuning of those models.

You and I have talked some of the best products out there in the enterprise we call low friction in and high friction out. Right. And to the extent that you can lower the friction to sell Instabase or whatever AI platform to the enterprise, quicker, easier, easier they get them using it. And then the moat is not in the fine tuned model, the moat's not in the workflow the applications are building. But the whole idea is to reduce the friction to get into the company. And sounds like using these large language models eliminates a lot of the friction that fine tuning will present.

And also we can solve any use case. Right? Because I don't have to worry about this problem or that problem. We just tell you, do you have any problem much easier.

So you just launched AI Hub, which is the updated version of the same platform, more or less, but now it's powered with large language models, correct? That's correct. So tell me, how does AI Hub work with the rest of the Instabasis products?

So AI Hub is built upon under the hood, same platform, the same distributed operating system which we started with. But when you have a technology shift like this, like when you can do zero shot, that means zero shot training.

Is what you're saying.

Yeah, and they're not training zero shot answering it. Actually that means the interface has to be designed for that simplicity. Now earlier you need to fine tune, you could not ask a natural language question, now you can do conversation. So we basically rewrote all of the UX of AI Hub so it looks like a completely new product. And that UX is very inviting, very easy to use, anybody can use it. So you can go to aihub.in Sign in with your Gmail, put any documents, ask any question, build any application, publish that to our app store and it's so easy.

What about all the other little Instabase apps and tools I had before? Are these also subsumed to the LLM or are they still part of the product because they're still in the same operating system?

Yeah. So for complex use cases and enterprises we still provide those tools because they are still needed. And we are rewriting much more simplified version of tools that can use LLM. So we will still have refiner, but LLM based refiner. We will still have validation, but it could be that LLM can do the validation. You can still write UDF if you want, but LLM can do the validation. We will still have human review, but much simpler human review. We are just rethinking entire product. Especially given the simplicity the underlying engine provides. It has to provide similar affordance to those users because earlier it required like it was designed for some business operations ops team who will take data annotate, then somebody will go build the workflow. Now we just think everybody should be able to self serve and that is possible. So AI Hub is built on the same platform, has all of those wells and vessels, but many of them are hidden and the platform exposes much simpler, easy to use version of the same things.

And part of it is this chat interface, what we call the system engagement. I can now interact with my docs like GPT, ChatGPT or Bard, et cetera. I can now do that, interact my apps, but also not just interact with my docs. I can use this chat system of engagement to actually start building apps, right?

That's correct, yeah. So basically actually.

So instead of drag and drop, it's almost drag drop in chat.

Yeah, yeah. So for example, sometimes you just want question, right? You can put all the term seats of Claylock and say tell me all the terms where this particular provision is true. You don't have to build apps. You can just ask that question gives you the answer. And that is much simpler affordance, right. Than earlier thinking of like training a model to train that piece makes no sense. So yeah, so one of the affordance that we provide or the interface that we provide is called the converse. That means you can have conversation, but this conversation is not like the knowledge from the model, but the conversation is knowledge from your data so that it is factually correct and you can have that conversation and you can get answers from your documents and your data and.

We also used another Greylock investment, Llama, and actually kind of ingest some of the data as well to interact with, which is great. So does this widen the number of problems instabase can solve now? Because now all of a sudden you can access all my data. You have the power of these large language models. What does this do to the original mission of Instabase from 2015? Now we're in 2023. What's happened to that mission or that goal? Is it bigger, broader, different?

Our mission remains exactly the same. That has not changed a bit because our mission was to build the ecosystem for humans and for individuals and organizations to build these applications. And that's exactly the same mission. So along with Converse, we have build. And along with Build, we brought the old app store. But now these apps can be built using LLM and, and can be published to our app store.

So I can still build these apps using AI Hub.

Exactly.

But now. So just like all the apps I built before in Instabase, I can do. But now it's the enterprise way to use lm. So if I'm an enterprise and I want to use GPT or Llama 2 or something, the enterprise, I can use Instabase instead. So that's fascinating, but these open source LLMs exist out there like llama2. There's also anyone can go to OpenAI and get an API to GPT4 GPT 3.5 Turbo. Why can't I build this myself as an enterprise customer?

Great question. I think three different reasons. Number one is all of these language models or over API, the way they work is you give some context, sequence of text plus some prompt and it gives you another sequence of text. So unless you can represent your problem exactly that way, it will not work out of the box. So for example, let's say I want to build an application for loan processing that has a classification, extraction, layout awareness. So there is a ton of engineering that has to be done before you can use them.

That's a great insight. It's not necessarily the models are limited, but how you represent the problems. The models.

Exactly. I mean, there was some famous scientists who said solving the problem primarily is representing the problem the right way. So now these LLM have certain representation domain and you have to figure out how to represent your problem into that domain. And if you cannot, you have to bridge that gap by doing a ton of engineering. Right, that's problem number one. Second, these models have some limitations, right? The limitations of the context window, like they can only take 32,000 text or now 100,000 tokens and so on. Second, they can hallucinate, right? One of the problems that you're seeing because they're, because these large language models are trained on the Internet data. So they become very good at two things. They learn the reasoning, of course, first. Second, they also compress all the knowledge from that Internet into that model. That's why they're able to answer those questions. That is a lossy compression. Just think of like all of the information on the Internet compressed into that, like some few terabyte models. Some relationship gets muddled up, some stuff gets represented in a way that it cannot retrieve. And that's why you see the hallucination. So what we end up doing is in order to solve that problem, we don't use these models for knowledge representation. We represent the knowledge completely outside the model in an uncompressed way. So that way you basically can reason on the data that is not like compressed in a way where you lose a lot of information. And that will guarantee that you're getting factual answer. You're not getting an answer trained on Internet. Like one really good use case. And we were talking to a person who does mental health counseling and he wanted to provide like all his customers ability to ask questions. The issue with ChatGPT is like there's so much misinformation on the Internet, right? So if you ask about something, it can give you some answer that might be not correct. So what that person ended up doing was he took Instabase. The person had written 10 books and the notes from last 10 years. He put all of that inside Instabase and now built on top of it. Now he could scale himself, right? Anybody can ask question. It guarantees that those answers are factual and those answers are exactly the answers that he would give because he wrote those in his books and in his notes. So those are the things that basically you have to really understand that LLMs are a great tool, but they are one part of this big product that allows you to solve these problems at scale.

That makes sense. So problem one was defining the problems. Problem number two is separation of almost duties, where you have the reasoning from the models, but the data, the factual information separate that you handle. What's the third reason?

The constraint that these models basically have. For example, these models only represent languages sequence of text that might not be true. How do you understand two dimensional information like documents? How do you understand images? How do you understand things that cannot be represented as sequence of text? So there are all of those things that basically has to be Solved for these workflows to be created. Let's say if I want to do lending, I have to do classification first. Before that I have to do ocr, then I have to do the extraction, then I have to do the business logic. These models are not very good at mathematical problems so that all of those things have to be put together so the problem nature does not change. Where these models are very, very useful is ability to reason on given sort of context and that's where we use it. But eventually AI Hub is a complete suite of products that allows enterprises to.

Solve the problem, solves the problem, solves it, represent the problem correctly and it lets you get repeatability by building these apps.

Then we have App builder that allow you to build those and then create the enterprise App Store for usability where you can again modularly represent all that.

Tell me about this. So everyone has an App Store. What is this App Store inside AI Hub? Why would I use it?

So App Store basically represents a collection of applications or modules that that can be reused. So given that we are in the business of automating these complex business processes, so what Instabase was designed around is whenever you are creating these workflows, just think of them as a modular building blocks. And these modeler building blocks can become the underlying component of the higher level applications which can itself be shared. And as we saw that sort of like architecture working really, really well within our customers, within our large enterprises, it makes total sense to have that reusability across different customers too. And that's how this App Store evolved. So because we are a workflow platform, it makes total sense to have the App Store because workflow collects in a bunch of these components.

So now you have Instabase, that's evolved into AI Hub. So you had the best of Data Hub. The Data Hub DNA is still there, the operating system, the platform, the best instant base is still there. This whole idea of representation, the problem solving the problem, laying out for the customer. Now you've incorporated the elements of large language models. I would say you've done a great job evolving the company over seven years. And a lot of founders don't, right? They just stick to the one thing that works from 2015. You've done incredible jobs adapting along the way. So that's been the past seven years. What's in the future for Instabase next three, four years? Either product wise, tech wise or business wise.

So yeah, so tech wise is very, very hard to predict. Right? Technology. Nobody would have predicted like you know, this LLM would become Such a powerful thing three years earlier. But I continue to believe that these LLMs would produce become larger and larger, would become more capable. But our focus area is mainly around building the community and the ecosystem. So far our customers have been primarily enterprises, mainly large organizations. But we have launched a very community version of AI Hub which anybody can use. Then we have commercial version of AI hub that you can get started, SMBs and startups can start using it. We are investing very, very heavily in creating an application ecosystem and developer ecosystem because once you win that now you have so many different modules that are available that can be used out of box by a lot of people or can be used as a component for their bigger problems. And I think we want to solve that at global scale. So that is going to be our core focus area, which is winning the App Store.

App Store solving all the workflow problems around the globe.

That is correct.

That's incredible. But one of the most amazing things is it was a very complicated product, but you've not only evolved the power of Instabase, but you made it easier to use, which is hard to do, right, because most enterprise software gets more complex after seven years. You've done an incredible job of simplifying things to make it simpler. Simpler. How do you guys do that?

Just a quiet, very ruthless focus. I mean, in general, because you have to throw away old thing. And I think one of the things that when we started AI Hub and we said four things, number one is we are not going to bring LLM to our product. We are going to rewrite the product that is rightly designed with the correct understanding of how LLM should be utilized. So we literally completely detached from the past and rewrote the whole user interface so that we don't get attached with all of the UX complexity that we had built earlier. So that's number one. Second, I think this was the goal that I gave to the engineering team is focus should not be on how many features we build or whether this product is functional or not. That is expected that product should function. I think here we were chasing much bigger goal that is just the right product. And that was a bigger question. Is this the product that we can recommend to our friend and family? So the goal that was given to the engineering and product and design team were very, very, very different. Third one was, we call this like reductionist principle. Keep removing things until removal of that thing breaks a product. So anytime somebody added even a single like button, we asked this question, is it needed? And that's what led to the simplicity. And I think we'll continue to go in that direction. And this has been, I think a focus change that required. And it was hard, it was actually hard. But eventually I think like when the product came that simple, people were proud of that.

Well, I mean you have the three Rs you describe ruthless in terms of discipline, friends and relatives. You want a product that's proud for them and then reduce, reduce and remove stuff you didn't need to make it as simple as possible. So ruthless reduction and make it proud to show your relatives. But a lot of companies don't do that, right? Like famously Microsoft in the 90s tried to shove Windows into every platform, mobile, tablets, whatever, and Apple said we're going to use the right operating system for the right device and they build iOS versus MacDonald. A lot of companies try to force fit a square peg into a round hole. You guys said, okay, here's this incredible round hole part of my torture analogy, which is opportunity with lms, you built the right product with the right hole in mind. The right fit, the right product, the right problem.

We tried. I think there is still a scope for making even simpler. I think you will see the newer version that's coming that will force even more simplicity. And the fourth which is most important was we basically would not do many half baked things. We'll do a few things done really, really, really well. So in fact we ended up not releasing many of the features that we deeply cared about that was part of the release and we said those are not good enough. So in fact you see like multi file support, the ability to do retrieval, the ability to do like human review and refinement, we did not release those as part of the first AI Hub because we did not feel it was at the point where we could be proud of what we built and we could recommend to our family and friends. So these were the four key ideas behind how we approached AI Hub.

So that's just a reflection I think of your leadership as the founder CEO and kind of their product discipline. What other lessons the past seven, eight years have you learned as a founder CEO running Instabase?

Yeah, I think it's very, very hard because there are just so many lessons. I made more mistakes than I think anybody else in the company building. But one of the key lessons that I think could be useful for others and definitely that's something that has been useful for me and I keep reminding myself is it's very, very, very hard to predict the future and we should not be in the business of just Predicting the future. I think the right way to approach company building is being curious and fearlessly experimental where we can engage with the world in the right way, where we can engage with the world with curiosity that what is changing and how we can adapt to it. It's very, very, very hard to predict the world. Right. Every single time.

So just stay curious as a, as an individual, stay curious as a company and fearlessly experimental.

Because sometimes we get like, this is a mature product we don't want to touch.

Yeah.

And that could be problematic.

So fiercely experimental.

Yeah, that's great. That's one second. I think many founders and CEOs are doing it for the first time and they basically come from academia or they have never even worked at a company before. And sometime that basically gives the sort of feeling that once you hire exec for everything, your problem is solved because they are expert in their field and you let them make the important decision about strategy, about how to run business. I think as a founder and CEO, it is important to have really, when an exec make a decision, it's important to ask why are they making the decision? Don't trust that they understand the world better. Because generally if you don't understand, you realize very, very late that something has gone wrong and you have no ability to fix it. So being very deep, detailed, even after exec hiring in every single aspect of the business, it's critical. You cannot sort of offload that work because you hired an exec.

Well, I think in all situations and conversations, seek first to understand is a good rule. Right. Both in decision making, but also I find when people have disagreements or arguments is because we don't take the time to seek to understand. Right. And I think you can trust everybody, but I think seek to. I want to come to every conversation with a learner's mindset and I want to enter every engagement with you or whoever else with Seek first understand.

Yeah, yeah. And the last one, which I think is not less and it regardless of. And maybe some people don't make mistakes. You'll make a lot of mistakes. There will be things that will go wrong several times and you just have to. That's part of the job. You have to live with that. And I think getting sad because we made a mistake and we lost 5 billion, $5 million. Getting sad did not solve the problem. It's basically what solves the problem is fixing it, making the right decision and keep doing it. And if we end up succeeding, great. If not, just go do something else.

Well, thank you for sharing these lessons anant with myself and obviously the Gray Matter audience. And thank you for being here with us today.

Thank you so much for all the help. And thank you to Greylock for all the help, because Greylock has played critical and an important role in shaping where Instabase is today. I'm very thankful for that and hopefully we'll build something that we all can be proud of.

Thanks, Anat. You already have and we're excited for the next 10 years of the company.

Thank you so much.