Beyond Coding: JavaScript Full Text Search, Entrepreneurship, and AI | Michele Riva

Welcome back to the Code Ryan Podcast. Today I have on— I'm going to just call him Mitch because I butcher his pronunciation of his name, but he'll definitely give you the proper pronunciation here in just a second. But Mitch is a very experienced software engineer and architect with 10+ years of experience. He's a Google Developer Expert, Microsoft MVP, and is from Milan, Italy. But he is kind of coming over to the States here shortly, which he told me. Kind of the intro of this podcast as well. But not only this, but Mitch published a book in 2022 titled Real World Next.js, and he also co-founded Orama Search, where he is working as a CTO building the new generation of search engines. So welcome to the show here, Mitch. If there's anything I missed in that intro or anything you want to correct, definitely let me know.

Yeah, thank you for introduction. I won't be pronouncing my name because I feel like it's pretty strange and I I feel like we should call myself Mitch. It's, it's totally fine. I get it. No, you cover everything. So that's, that's who I am. That's what I'm doing. And yeah, I'm very happy to be here.

Perfect. Well, today I was hoping to kind of dive into, you know, what you guys are doing at your company and what kind of problems it solves and maybe even getting into kind of some of the technical aspects of it as well. So I think a good place to start is first, like, What's Orama Search and then what kind of problem is it aiming to solve?

Sure. So Orama Search is a company I co-founded with other professionals and we are aiming to build a new generation full-text search engine that is willing to compete with great competitors, of course, in the space of full-text search. We noticed that full-text search is, yeah, it's something that everyone needs. It's one of those things that you don't realize how much you actually really need them until you don't have them and you need to evaluate what's the right stack. So you might have, I don't know, Algolia, for example, which is a beautiful product. You might have Elasticsearch, which is also wonderful, MeiliSearch. There are plenty, but they all approach the same problem with this, with in the same way. Our idea is to approach it in a totally different way by deploying basically our solution to a CDN. So we are globally distributing your full text algorithms and data, etc., on 300 global locations in 100 countries worldwide. And there is very low latency whenever you have to make a request. Performances are most of the time two digits in round trip. So whenever you make a request, you got your response in less than 100 milliseconds on average. And performance is incredible. It's built in JavaScript, so you can easily extend it and, you know, do whatever you want. We do expose a hook system that allows you to hook into the indexing and search process. So you can do basically whatever you want. It can be your search engine. It's fully open source. You can fork it, you can hook into the processes, you can do whatever you want, and we take care of the distribution. So that's what we do.

Awesome. Well, that definitely sounds really cool. And I think maybe to even take a step backwards now, if someone isn't familiar with what a full-text search is, maybe we could cover just like generally what is full-text search and then also what kind of got you interested in it. Well, like, why was this a problem that you felt like you wanted to be involved with kind of going through solving here?

Absolutely. So when we talk about full-text search, we do refer to a procedure of searching a series of tokens inside a large document set. So let's take an example. Let's pretend for a moment that we work in a library and we have millions of books and we want to retrieve all the titles of all the books that contains the word star, for example. We could take a linear approach, meaning we take one book, read every single page, and then if the book contains star, We save it apart and go to the next book. But of course, if we have 1 million books, we have to read 1 million books, right? And every single page for this book. So it's incredibly expensive in terms of computation and performances. So full-text search engines do solve this problem by adopting very clever algorithms and data structures that are built for this purpose specifically. And Orama, for example, which is the search engine we are developing at Orama Search, of course, is basically giving you the opportunity of searching through millions of books in just microseconds. So microseconds is a millionth of a second. So it's less than 1 millisecond. If you just, if we make the math, it's incredibly performant and it's building JavaScript, which is crazy. So whenever you go to any commerce, for example, then you look for, I don't know, black shoes, for example, you got two tokens. So you make a research for all the shoes that are black. If you look for, I don't know, iPhone case, you will look for all the cases that are meant for an iPhone, for example. So this is what full text search does. It looks through an incredible amount of data and gives you back the results that you do expect when you're making the search request.

Very cool. And what kind of got you interested in solving this problem? Like what? What? Stood out to you as, you know, hey, I want to dive into this and I want to make sure that full-text search is, you know, super optimized and that we just kick ass with full-text search. What kind of got you interested?

3 to 4 companies ago, I was working as a consultant and I was contributing to Apache Unomi, which is a customer data platform, and it's open source. And of course it's an Apache product and it basically used Elasticsearch as a leader database in its infrastructure. And we used to throw millions and millions and millions of data every single day against it. And while we were throwing millions of data, of fresh data to Elasticsearch, it kept the performances perfectly. Of course, we had some problems. For example, it became very expensive. Performances had to, you know, were a bit reduced, so we had to scale, but if you know how to scale the product such as Elasticsearch, performances are incredible and it felt like magic. So it's, it's incredible. We were having this customer which was an e-commerce customer and they were selling, I don't know how many thousands of items every single day. So we had to keep every single sold item inside of Elasticsearch. So whenever we made a search for, I don't know, for example, t-shirt, it took like milliseconds to retrieve all the sales for t-shirts from the start of the day till the night. And it was incredible because we had millions of data. So that's what got me into full-text search. I was wondering, how is that possible? I don't have a degree. I don't have, you know, the kind of formative experience that can give me a hint on what's going on under the hood. So it felt just like magic and I wanted to learn more. So I had the opportunity in my last company I was working at, NearForm, it's a professional services company. I was part of the developer experience team and I had the opportunity of applying to a conference called We Are Developers. It's the biggest conference in the world for developers. 8,000 developers from all over the world meet in Berlin once a year for that conference specifically., and I had the opportunity to talk there and I was like, yeah, I need something special for this conference, right? So I will build a very small full-text search engine and let's see what happens. So that's how I started basically working on that, that later on became my company. Cool.

Well, it sounds like, you know, you really just saw kind of the magic of it and wanted to kind of be like, hey, how can we pursue this and make this even more magical? So I think kind of with that setup, when, when diving into this problem, you mentioned that you didn't really have like, you know, formal experience with full-text search and like you didn't go to school to, you know, build these full-text search engines and stuff like that. Maybe what are, what are some problems you ran into when trying to solve this and what kind of maybe even technical concepts did you have to better understand when getting into FullTech Surge, and what are maybe some interesting things that you kind of learned throughout this process of building your company and kind of implementing solutions to this problem?

So one thing that was definitely missing on my side, um, was the knowledge of the data structures that I needed to build such a product. So talking about the fact that I don't have a degree, I saw many of my former colleagues with a computer science degree saying, yeah, of course it's fast. It uses, I don't know, a Radix tree, uses a HashMap. Of course. Yeah. Okay. HashMap, maybe it's too easy. Everyone knows that. But I know an inverted index, an AVL tree for numbers, whatever. So, and I was like, yeah, but you know, as web developers, I was a web developer back then. I don't typically use this. I don't care, but eventually I had to care. And what about algorithms? Like, yeah, binary search, or I don't know, how to properly sort and insert data into arrays, how to compute the union or intersection of sets or arrays. Like, these algorithms are very hard to implement and very hard to kind of understand if you don't have a formal education in computer science. So that was the most difficult thing I had to face when building this product. And I'm glad I did because it's incredibly satisfying to deep dive into the rabbit hole of data structures and algorithms. And so that was the most difficult thing I had to face. Talking about, of course, as I was saying, I was working at NearForm and NearForm was pretty famous in all around the world in the Node.js community because NearForm is still today one of the top maintainers of the Node.js runtime. NearForm hosted people from, you know, many big communities nowadays as employees and they were developing Node.js, they were evolving. The commitment that NearForm had in making Node.js available to everyone was incredible. And so they were very committed to open source. And when I built, so the first iteration of my full-text search engine was called Lira. And when I built Lira more than one year ago now, oh my God, time flies. I got contacted, of course, by people that said, yeah, you could build a product around that, right? Why don't you do that? And I was like, I'm not a business person. I don't have the right team now. The right team is here in NearForm, but I can't just steal people from NearForm and get started, right? But eventually I talked with the founder of the company, which is called Kian. He's an awesome guy. And we had a very deep conversation, a very, I don't know how to say, honest conversation around that topic. And he later became the first angel investor in the company that nowadays is Orama Search. And he let me basically co-found this with the two other people from NearForm. And he also helped me reaching out to Isaac Roth, which is nowadays our CEO. He was the creator of OpenShift, creator of other great open source technologies such as StrongLoop, was one of the first to commercialize Node.js back in the day, and was also CTO in IBM. So a strong, businessman, a person that truly understands how to build and scale startups. And I am incredibly humbled to have him as a CEO and co-founder and business partner in this adventure. So it's all thanks to NearForm. This is how basically we launched the company and we launched it in, yeah, around March in San Francisco, California. And yeah, I really hope to be able to move there pretty soon because I've been there talking with investors and talking with people and it's an incredible place for everyone doing tech. That's really it. It's incredible, especially for people like me coming from the province of a big city, but I'm not from a big city. I'm coming from a small town. It's like 4,000 people here getting into big cities. It's a dream come true and it's incredible how technology can bring you that far.

Yeah, absolutely. That, that's definitely awesome. And I know that— so I grew up in a very, like, small city. I think we had 5,000 people in our city and we had, like, you know, a total of 3 stoplights in our entire town. So I come from a small, like, farm town in the Midwest here in the States. So, like, tech enabled me to move to, like, a big city and, like, kind of experience that. And it's definitely been a super beneficial thing to my life as well. And You know, hearing your kind of story there, I know that for myself personally, and probably for a lot of people listening, maybe they're more technical, and they're probably a developer, or they're software engineers, something along those lines. But maybe they also have an idea for a product, and they want to start building your company and kind of go through that process. And so I was like, you were kind of in a similar spot to where you were technical, but maybe didn't have the experience with business building and stuff like that. And we're able to kind of find a really good partner and to kind of get going with that. So maybe, I know we didn't necessarily talk about this beforehand, but would you have any advice for someone that is maybe a developer or someone that's more technical and wanting to kind of start their own business and make their kind of idea come to, come to light? What What kind of advice might you give them for kind of starting their own thing there?

Yeah, sure. So first of all, it depends on the kind of business you want to have. If you have a side project and you're sure that it won't affect your working hours. So let's say you're working at Google, for example, let's just make an example and you have a side project and you do that at night. So you're not stealing working hours from Google, if you know what I mean. So if you work on that and you can grow the business while you're working at some other tech company, that is perfect. Whenever you feel comfortable, you will have your paycheck at the end of the month or like, yeah, maybe in the States twice a month. I can't really remember that. I'm from Italy. It's very different here. But yeah, so in Italy we got it at the end of the month, right? So whenever you feel comfortable enough, to be a sustainable business that can sustain your expenses month by month, then you can quit your job and get there. There's nothing wrong about that. You don't need anyone else. You just have your business. You will start hiring people little by little, and eventually you will evolve and scale your business. Another way is, of course, going to venture capital. And especially from, if you're from San Francisco, if you're from California, all the big tech companies we know today had to access venture capital funds sooner or later. And I have this belief that I've been pretty lucky in that because of course I'm partnering up with Isaac and he knows the game. He has experience in that field. So that's good. But I also had the feeling like, and now it's basically the leadership of Arama, it's me, Isaac, and Angela. Angela is our head of design and co-founder in Orama Search. Neither me or her, we are not business persons, right? But if we were alone together into that, we could have been in a situation where we were asking basically for funds to venture capitalists. And I have this feeling that they might understand that we we lack of certain knowledge, they have the patience most of the time to understand that we need some guidance. And that's something I do recommend to people whenever they really want a business that scales fast, that generates profitability very fast. You can go to venture capitals, for example, ask for funds in exchange for equity, of course, but also guidance. They can provide you people that can guide you and mentor you. And of course, it's a trade-off. At a certain point, you will leave part of your company shares to this venture capital, but you also have to understand that this is one of the companies you're gonna build in your life. So maybe this is the test ride, and then you build another one and another one and another one with the correct experience. This is what I like about the States, by the way. Like, this is not something you can— it's not the kind of reasoning you can do in Italy. You can do that in the United States, and it's amazing how you can do that in 3 days, building your company with Stripe Atlas or AngelList. 3 days, you are ready to go. I would recommend this to every single one of the audience because it's a very fast and effective way for building a company in the States. And then you just go out and seek for business partners. Especially in California, I see there is a lot of people willing to help you in exchange, of course, for paychecks, stocks. Then you decide. You, you are free to make your own mistakes and find out what you did well and what you didn't.

Yeah, absolutely. Well, I think that's great advice. I think, you know, depending on someone's personality and the specific business and product that they want to build, one of those approaches might make more sense to the other. To where for some people maybe it's way too risky to go take on venture capital and then, you know, try to build your business that way and like put rocket fuel onto it with a bunch of money and stuff like that to where they might be more comfortable with just like, you know, as what I'm doing right now before work, recording some podcasts and kind of do the content thing and then kind of working on the side of the job to where, you know, others might really like the aspect of going the VC route. And then also you mentioned good point on they also give you some more guidance there to where they might give you like, hey, this is how you set up payroll, or this is how you should approach doing this or that. So I think there's definitely pros and cons in each. And depending on the product and someone's own personality, it makes sense to kind of go one way or the other. But to kind of flip this back to around the search here, you mentioned using JavaScript for this. So I'll be interested in, you know, why you decided to use JavaScript. You know, some people mention that JavaScript's slow and that you should just use Rust for something like this. So what, why did you kind of choose JavaScript and what was the reasoning for that?

Yeah, so first of all, as I was saying, I was working at NearForm, so we were core contributors to Node.js. That was the language I knew. I wasn't actually contributing to Node.js, but my team was. So the expertise that we grew, we grew, it was all around Node.js. That's the first point. Second of all, I'm a member of TC39, that is the international organization that standardizes the JavaScript programming language. So my expertise, once again, relies on JavaScript. One other thing is that, in my opinion, that's the biggest learning I take from building such a product, is that, in my opinion, there is no slow programming language, there is just bad algorithms and data structures design. If you now go test Orama open source, so the one you find on GitHub at /oramasearch/orama, you will notice that you will likely get results in microseconds again, or milliseconds. It's very fast. And the reason why it's so fast is because we know how JavaScript works. We know how, you know, how to make it fast How to make it comparable with other existing solutions that maybe are built in Java, C++, Rust. And of course you cannot compete on raw performances with C++, with Rust, with Java, but you can compete at the infrastructure level. So in JavaScript, there will always be a point where you hit a performance wall. You can't go pass this wall. So you have to get smart and create an architecture that can help you compete with other solutions. In that sense, software architecture and software engineering need to play together, uh, for that specific reason. Because if you match them well enough, then you have, um, a product that can really compete with other products. And we see that, for example, in the enterprise, If you had now, let's say, Guillermo from Vercel, you might ask him, why are you building everything in JavaScript like Next.js while Spring Boot from Java was the standard back then for, you know, creating enterprise applications? But he would probably, I don't know, so take this with a pinch of salt, but he would probably say, Yeah, but you know what? Our architecture allows you to do that on Vercel. That's a perfect example to me because there is no bad programming language. You can do incredible things in PHP, like Facebook is still written in PHP, right? You can do incredible things in Python and Ruby if you really know the language underneath and if you are a good software architect. That's my learning from building Orama Search. Because I truly see that every single day when, when we are measuring performances, memory occupation, wherever, it's clear that if you know how to optimize for language, you can get great results.

And, you know, that definitely spurs the question of if someone is a JavaScript developer building these applications, is there any common mistakes that you see people make when using JavaScript and maybe leading to JavaScript being a little bit slower than it needs to be, or what can someone maybe do to better optimize their JavaScript or better optimize the runtime that they're kind of using?

Yeah, so first of all, if you're using Node.js, for example, I would highly recommend taking a look at Node Clinic, which is an open source project that was built by NearForm, and it, it's basically a memory profiler. So whenever you, let's say you create an Express.js or even better a Fastify server, and you start having tons of requests, you can use, let's say, a flame graph to see what is the function that it's taking the most to, and it's slowing down the whole process. So you can concentrate on this function specifically, try different approaches and see what's going wrong. Then I like to say that I'm a former functional programming guy, and I gotta say, I really love coding in Haskell, in Elixir for a tiny bit, and other functional languages. So it was clear to me that functional programming, it's a beautiful way of, you know, composing function and software, but it, in my opinion, it's not applicable, applicable, sorry, to JavaScript. So if you have like memory intensive usage inside your application, like, functional approaches are not good enough yet in JavaScript, in my opinion. Recursion is slowing down everything. Every single function call, it's giving you some kind of overhead in execution time. So it's not really worth it. But there are places where it's not a problem. Like on the front end, you have one machine for one user, no problem. On a consumer, let's say you're consuming a queue from RabbitMQ or a message from Kafka, you have one consumer for one message. But if you have a server, so let's say a Fastify server, web server, you have one machine for many users and you want to optimize every single request. So now here is where you might want, not an object-oriented approach because this isn't working either, but an imperative approach. Good old imperative style. That's the number one way in my opinion to speed up everything. Then you have to learn algorithms and data structures. Then you have to learn how to optimize for your runtime. If you're using Deno, if you're using Node, if you're using browsers, if you're using Bun, every single runtime has its own optimizations. There are many ways to optimize them. Just look on Google. You can find whatever you want on Google. Don't trust ChatGPT yet on this one. Go on Google, read papers. There is a lot that you can learn from Stack Overflow still. And I do highly recommend doing that.

Awesome. I think that's really good advice. And yeah, in the span of a 30-minute podcast, we definitely can't talk about every single little thing you could do to optimize your JavaScript runtime and stuff. But I think that that was definitely some good high-level advice there and then also some good places they can go for that. So with that, you— and I also did mention this prior, so just let me know if you want to go different out here, but You mentioned ChatGPT there, and there are a lot of developers that are kind of worried about ChatGPT and the effect that that might have on their jobs. And, you know, maybe this AI is just going to start kind of writing these web applications by itself and we won't need developers anymore. So do you have any, like, opinions either way on whether developers should be worried about ChatGPT or whether it's just like a tool that they can use or How are you kind of feeling about the state of AI and how that affects developers?

I believe eventually the code will be writing itself. So that's clear. Like Copilot made this very clear from the start. And I love Copilot. I love ChatGPT when it works. I have no problems with this technology. I was talking with a friend of mine on another context, talking about copywriters. So he was trying to hire a copywriter and he made a test. So one copywriter for a— he is a consultant, so he was trying to sell a copywriter to a client. One copywriter could have used ChatGPT. So it's basically copywriter plus AI. Another one cannot use ChatGPT. And then the last one, only ChatGPT, and then see what's going on. Eventually, of course, they found out that number one was the per— the client preferred the version where the copywriter was also using ChatGPT because it's clear that ChatGPT itself won't replace anyone right now, but the people knowing how to use it properly will eventually take more space in the competition, right? So right now, and the thing is everything is moving so fast that whatever we're saying, it's just pure speculation, right? So my opinion right now, 2023 in July, is that eventually we will get replaced in some ways. But for example, if you're making, let's say, if I think of how I made Orama, an AI could have never have thought of something like that because no one else have done this in the past. So we still need, we don't have a solution that creates new solutions. We have solutions that basically knows a lot of things and can help you with your tiny problem and can help you one problem at a time. So it's iteratively helping you to solve your problems. But when you need to create something new, you still need the human advice, I think. And it's going to be like that for a very long time. So there will be people taking advantage of all these new technologies. I will be one of them because I believe they are just beneficial for me. But there will also be people, and I really hope to be one of those people, of course, that will create something new that AI still cannot create when it comes to architecture.

Yeah, absolutely. And I, I think I lay in a very similar spot to where I don't know where I heard it, but it was like something to the effect of, you know, AI is not going to take your job. It's going to be the developer using AI that will take your job. So currently it is a tool that developers can use to kind of, you know, supercharge your skills a little bit and improve their productivity. But as you said, you know, maybe a decade from now we have these auto copilots that just, you can give them a prompt and they go and, you know, they consume your entire code base and then they go add different components and stuff like this and do a lot of that kind of auto magically. So I think that's coming at some point in the future, but I think for a good while here, they're still gonna be— yeah, yeah, honestly. Um, but I think that for a good while here, developers that use ChatGPT are still you know, in the clear, but still you need to stay up to date on these tools or else someone else is going to use them and be more productive and it's going to probably hurt someone's career growth there. So yeah, I'm very much so in agreement there.

Yeah, I know there is a lot of, let's, let's call it machismo from certain developers saying, yeah, I don't need that, I'm too good for this. Yeah, like no one is forcing you, right? Do whatever you want. I feel like I need it. So I will just need it to be fair. Like, there's no, I do believe that we faced a similar problem back then when Google arose, you know, so everyone can be a developer, so everyone can steal our jobs and no one ever touched, you know, all developers' jobs because they are all still employed or they eventually retired, but they just created a new job that's That's what I'd like it to be. And eventually, I believe this is the direction it's going to take.

Yeah, absolutely. And, you know, at— on some level, there's all the problems of the world are definitely not solved. And there's a ton of new problems that are going to be needing to be solved. And those problems are going to involve technology and developers building technology. So I say, you know, learn how to use these tools. It's probably going to make your life a heck of a lot easier as a developer and make you more productive. And we're probably a ways off from just a, a general artificial developer that does everything. So, you know, learn these tools while that's available to you and do the best to kind of stay up to date with everything. So I think with that, Mitch, we kind of went off in some different directions for this podcast. We talked about business and AI and stuff like that, but I think we covered a lot of really good stuff here, especially regarding your company and kind of the optimizations it uses with JavaScript and different things like that. So if people want to stay up to date with kind of what you're doing and, you know, want to stay up to date with any talks that you give, I know that you give several of those as well. Where can they kind of go to make sure that they're following you and kind of staying up to date with all that stuff?

Right now, I would say on Twitter. But I'm not sure how long will it last. So you can find me at Michele Riva Code or on micheleriva.dev. I will make sure I will update my social links as soon as something, you know, that it's working will come out. Maybe Instagram Threads, I don't know. But yeah, I'm a hard user for Twitter, so I really wish it could work properly and talk to everyone here.

Awesome. Well, thank you so much for coming on today. I really enjoyed it.

Thank you for having me. It was a pleasure.