E384 Deedy Das, Founding Engineer at Glean

The Aldis podcast is brought to you by Aldis International, supplying your expert AI and digital transformation staffing needs across the US And Europe. Today you are listening to our AI in Action series where leading minds in AI from across the world share their story, success and advice. AI in Action cuts through the hype and explores the true impact of artifact artificial intelligence in our world. Today.

You'Re listening to AI in Action. I'm your host, JP Valentine. Our guest today is Didi Das. Didi is a founding engineer at Glean. Didi, welcome to the show.

Hi, nice to meet you, jp.

Yeah, you too, man. And thank you for coming on to do this. We're very much appreciated. So, Didi, let's start with yourself, please. Could you give us a bit of background on your journey in tech from where you got started, some of the roles you've held along the way, and take us up to today as one of the founding engineers at Lean.

Absolutely. I started my tech career in New York at Facebook right out of college, and I spent about a year there where I was working on some search stuff, but a lot of stuff related to places I didn't particularly enjoy Facebook. And then I moved into Google where I was much more excited by the problems around search and how defining that is to the era of the Internet. I think the search brought the Internet to everybody. And after about four years at Google, where I spent some time both in New York, Israel and Bangalore and a couple of other offices, I thought I got a really good feel for it. And I think I was ready to start or be a part of something really small on my own. And then I joined Glean when we were about five to ten people, super small. And at Glean, we do enterprise search, so we take a lot of the learnings. A lot of our initial team comes from a Google search background. And we build search for the workplace, which might sound similar, but is an entirely different set of problems.

So thank you for that. You've already touched on what Glean does at a high level. Follow up to that, then walk us through the differences. Why would a company use Glean as opposed to more traditional search? And then as one of the engineers, give us some insight into the advancements that you guys have made on the engineering side and why it's impactful for your customers.

Absolutely, yeah. So enterprise search is very different from consumer search for large set of reasons. One of those reasons is when you're looking at consumer search, everyone has access to all the documents in the web and there's so many documents that when you're looking for something, there's typically like hundreds or thousands of documents that might suit your need. So you're solving a little bit of a different problem. You also have access to millions if not billions for Google of click feedback data. So when you're designing ways to rank results on the page, you can build models that are trained off of billions of click feedback data for even the same search to make your algorithms a lot better. And that's a much, I would say, easier way to solve the problem compared to enterprise where you have two important variants. I'll talk about. One is not all your content is accessible to everyone. You know, you don't want everyone to search through your Slack DMs. You only want to be able to search through the to through messages that are visible to you. So permissioning is one element of complexity. The second element of complexity is you just don't have access to to that level of feedback data. If you're looking at a company, even a company of 100,000 people, if everybody uses your search and they search for something five times a day, it's not that much data compared to what in consumer search. The techniques you use are very different in order to solve that problem, not to delve too much into it, but for example, one way to counter the differences in the problem statements is to leverage the relationships between people at the workplace as opposed to consumer search, where you don't know much about the user really aside from things that they search for. In an enterprise setting, you actually know what does this person do, who does the, who do they report to, who do they work with, what kind of documents are they used to accessing? And that really helps you make relevance a lot better for us in enterprise as Glean compared to traditional consumer search engine.

And then as an engineer, I would love for you to speak to our audience who, many of whom are engineers. Can you give us some insight into what it's like behind the scenes, day to day, what it is that attracted you to the work at Glean? And just give us a bit of a look behind the scenes of the day to day if we were members of the data and engineering team with you.

So what attracted me to Glean is there's many things, but particularly technically I will talk about. I'm just an absolute nerd for search. I think it's an extremely underappreciated problem. Even now you have a lot of people who critique how good even Google is. But search is a problem that's not really been solved unless you can say that no matter what you search for, you will find exactly what you're looking for, which is basically there's an Oracle, search isn't solved. And search is to me the problem of how close can you get to being that Oracle for every information need? And that's what attracts me to search. I think the problem statement is just so core to what technology can really do for humans. And now how does it look behind the scenes? It's actually much less sexy when you actually work on it. When you're working on search, I think as with many AI or ML applications, a lot of rigor is on process, particularly on our search and intelligence teams. When we work on ranking problems, a lot of our effort goes into answering the question, are we measuring the right thing? Are we evaluating this correctly? If the numbers show that an experiment isn't working, is it because it actually isn't working, or we're not able to measure it correctly? And how do we really iterate with those measurement systems in place so we can guarantee that in six months or a year when we've run 10 or 20 experiments, that all of them actually accumulate to have a real significant impact on the user instead of just over optimizing on small changes that may not be a much better experience for the user one year down the line.

You are listening to the oldest podcast when you're looking to scale your team. Or if you are interested in showcasing your company in a future episode, reach out today. Or if you're in the market for a new role, visit our website to view open positions www.alldis.com.

As one of the founding engineers, you play a pivotal role in creating the culture on the tech side, but also helping build a team and grow. I know you spend a lot of time speaking to potential candidates as they come in and join Glean. When you're sat down with candidates or virtually and you're talking to a data scientist or a machine learning engineer about the current environment and the roadmap ahead? What is it that you tell them about the work that you're doing at Glean that gets them excited about this opportunity over some of the other great companies who are currently trying to hire?

If you look at the core problem, we are at a position where we sit on the entire set of data that a company has access to in many cases. And when you have access to that rich amount of data, there's so much that you can do with it. In consumer world, there's a lot of problems you can solve for consumer users. A lot of information is in silos, is in companies. And what I tell candidates to really get them excited is how can you really unlock intelligence in the workplace? A lot of times it boils down to the really simple things. If you think about. A lot of people talk about remote work these days. You join a company and you're working at home, you're talking to your manager on Slack. It is very painful to navigate. What should I even be looking at? What documents are there? How do I do this small thing? Is this a stupid question? Should I really ask my manager this or is there anything else I can go to to answer this question? And a lot of people complain about these really fundamental issues. And what exciting to me and hopefully exciting to candidates about Glean is when you leverage an entire company's corpus of data, you can answer these questions and you can make people's work experience significantly better. A lot of our customers tell us after using Glean, I don't know if I can even be at a company that doesn't have this. I don't know how I worked without this business for. And that's the kind of validation that really encourages me and keeps me excited about that problem. And also I think search and Glean specifically is it's painting the Golden Gate Bridge. Like I said, it's never really done. You always can improve, you can always make things better. The space of information needs at a company is near infinity and we can always do better in certain domains to meet that need. And as long as that exists, we can make people at work just happier and able to execute their function.

Final question from me then. Didi, you talked about a lot of the exciting work that you're doing and potential for the space. When you look at the roadmap and what opportunities available to glean over the next year to two years, what are some of the things that you're working towards or you're most excited about?

So we are at a point where we've crossed product market fit and we know that Glean is something that a lot of at least tech companies really like to use. Our next stage in our challenge is really figuring out how we can go beyond that and nail larger enterprise companies. How do we get big old 100 year old companies that might be using data sources that are not slack, that might be using things that are not Microsoft Teams, how do we get them on board and solve their information needs? And you can think of healthcare companies, you can think of finance companies. Really allowing us to scale both horizontally to a large set of companies. That do various different things and may use very different data sources as well as vertically to companies of maybe hundreds of thousands of people. We have a couple in that range, but we really want to optimize our search experience to that scale. One quick reminder is that search is really not just about AI, it's actually a lot of different fundamental computer science problems together. Distributed systems how do we make the search fast? How do we make it accurate? And of course then how do we make the search results relevant? So as we try to conquer different kinds of customers and figure out how we can be valuable to them, it's really important to make the right abstractions in various parts of our technical stack so we can be useful and make search better.

Didi, thank you so much for coming on today. Really appreciate you talking to us. Great to learn about your own journey at Glean from founding engineer to where you are now. Search is always an exciting space and it sounds like Glean is really pushing the boundaries of what's possible and the success is showing with the customers that you guys are acquiring. It sounds like it could be a great place to work for engineers and scientists in the months and years to come.

Thank you jpm. Really glad to be on here.

Thanks for listening to this episode of the Aldis Podcast. If you enjoyed today's episode, don't forget to subscribe, rate and review. We are available on Apple Podcasts, Spotify and any Android podcast of choice. You can also head over to our website, www.aldis.com to listen to more podcasts, view our open rolls, and stay up to date with industry news. Thanks for listening and stay tuned for more great episodes coming very soon.