Build your own search engine, with Michele Riva

Welcome to the podcast.

Thank you.

Michele, is that the right way to say it?

It should be Michele, but for foreign people, it's Mitch, so it's easier for everyone.

Very convenient.

Exactly.

For listeners who haven't checked the links in the show notes, who are you and what do you do?

I work as a staff developer experience engineer at NearForm. We are a professional services company, and we are well known in the Node.js industry because we maintain a large part of the Node.js runtime. And yeah, we also maintain a lot of npm modules. Should be around 8% of the all global npm modules. It's like 1 billion monthly downloads. We are responsible for that kind of maintenance. Yeah, it's not bad. I'm also trying to help my colleagues to work on JavaScript standards with tasks that get selected by TC39. So, for those who don't know, it's the organization that basically creates proposals that involve the JavaScript language. So when they need like polyfills, when they need some community work, I try to staff people to work on this stuff, which is really, really important.

Yeah, yeah. To say the least.

Exactly. And at NearForm, I actually work full-time open source and I travel a lot for conferences and I've been working for the past several months on a full-text search engine written entirely in TypeScript.

Oh yeah, I saw you had some talks in the list about it.

Yeah. So yeah, that's how I got fun in, you know, these days.

Yeah, yeah, you had a really intense schedule for this year.

Yeah, like 50 conferences.

Yeah, so one a week.

Yes, one a week. Like this week I do two, for example, at Kølde. Because one today and one tomorrow. So you see, I can compress stuff.

Yeah, I think you have the hang of this. You've done it a couple of times. So I have to ask about the Node modules there.

Yeah.

How did that end up happening?

So, Neothron has been— okay, it's a very particular company, which I really love. And we used to have Matteo Collina, which for people in the Node.js community, he is well known. And he's part of the TSC, so the Technical Steering Committee of the Node.js project. He's a very prolific author of npm libraries. So, we ended up maintaining and helping him maintaining this these libraries because he was the chief architect at NearForm. So, he created, not alone, of course, but with his colleagues at the time, I'm talking like 8 years ago when everything started, he had created a culture around open-source software, especially in the Node.js community because that was our focus. So, I guess NearForm has been in a unique situation where people join because they want to run open-source projects. When I talk with people that know NearForm, so people that are really into JavaScript, they always ask me, okay, I want to join and I want to do open source software. So it's really attracting a lot of people.

So it's not like you sort of had to take over projects abandoned by other people.

Oh no, exactly.

It's sort of attracting that.

No, we also produce a lot of open source software.

I bet.

And for example, Lyra right now has like 5,000 downloads per week, which is which is good considering that it's a pretty new project. It has been open source in August. Oh, that's pretty new. Exactly. But I wanted to give a talk in Berlin about data structures and algorithms, and full-text search is a perfect example where you can find both. After I created a very basic library, Matteo told me, "Okay, you should open source this because it's working, it's fast, why not?" This is how open source projects are born. This is now part of the 8% of the code we maintain.

Wow. I've never written a full-text search engine.

Oh, it's really fun. I'd recommend you to do that. Absolutely.

Do you have any recommended starting points? Like any recommended reading or stuff like that?

I don't know. The problem is that I don't have any background that could bring me there. I don't have a degree. I'm self-taught. So, I started debugging Lucene or Solr. Started debugging there and here and a lot of talks. So, the guys from Elasticsearch, they do awesome job in making people comfortable in understanding how full-text search works. There are a lot of papers, so academic papers, you can read them and understand how it works. And eventually, it's really, really easy. The fact that it's kind of a situation where you don't know where to start because you don't have any direct experience on that. But once you start understanding that a full-text search engine, it's nothing more than a Radix tree, basically. And also, let's say you can start by doing just a regular HashMap, which is the basic, you know, you can use it as an inverted index, which is the fundamental data structure for a full-text search engine. Then you say, okay, maybe I can compress the data, I can do prefix search. So you transform your inverted index into a Radix tree. And this is how you learn. You see a problem, you fix the problem, and there's someone else that already fixed the problem for you. You just copy and try to understand how to make everything together to work. The nice thing that we got with Lira is that Elastic, which is the most powerful full-text search engine out there right now, for me, has been built in Java. You can exploit the Java virtual machine, the capabilities of the runtime, etc. You can't do that in JavaScript. We are kind of introducing a paradigm shift. When it comes, for example, to deployment, to maintenance, to defects. For example, here it's immutable. So, you know, it's kind of a paradigm shift but has its pros and cons, of course. And for the type of product that it is, it's just really convenient for us to keep it that way.

Yeah. You just made full-text search seem much more approachable than 5 minutes ago.

Yeah, the goal is for people to have one unique Full-text search engine, you can use it on the browser, you can use it on AWS Lambda, on your EC2 instance, on React Native. It's just one simple and common API amongst all the possible deployment stacks. So you just have this one, use it wherever you want, that's it.

I need to look into this. We might have a use for that.

Yeah, I'd appreciate any kind of feedback. Absolutely.

Wow. Where to go from there? So, yeah, one thing I noticed, or that you wrote, was that you've been working quite a lot, both as a consultant and at product companies. And that's, I mean, something that's been coming up regularly as a discussion topic in the podcast and among listeners as well, like this differences and similarities and what we can learn from consulting versus product companies and stuff like that. So I was just wondering if you have any interesting thoughts about—

Yeah, I would say I would add a third category, which is open source software, because right now I'm neither a consultant nor a products person. I'm an open source developer. So it's a total different category in which people need to understand that it's not all golden, you know, because it's like you have a lot of freedom, but it's not just things like that. But okay, let's start from the consultancy versus the product. Yeah, yeah. So I started as a system integrator actually, which is, let's say, it's just consultancy. At the end of the day, you do consultancy job and you're even more, um, like linked to the products you sell. One thing I heard, but that's my personal opinion, one thing that I didn't like about certain consultancy companies, and like NearForm, it's not one of those. We do consultancy, but it's, um, in a different way, and then we can get there. For example, I used to work for a medium-sized consultancy company in Italy. One thing I didn't like is that we had a fixed amount of products we could sell. We were specialized in those. And if you as a client come to me and say, okay, I need this, and we have a fixed amount of products, maybe they are not perfect for your situation, but we have to hack them to make them work properly. So that's something I didn't like about consultancy. If you do consultancy the NearForm way, for example, we do the other way around. You, as a client, come to me with a problem and I say, "Okay, let's analyze the problem. Let's find the best solution for you." We have, of course, a number of products where we find ourselves comfortable working in because maybe we are the authors of those projects. Most of the time, I find people to approach me because, "Oh, you are the author of that library and I need help." Okay, of course, I can present you to our sales department and then we can discuss.

Makes a lot of sense.

It's really convenient for us in that way. But if you have a very specific use case, like the COVID tracker app in Ireland, we made this. Of course, we couldn't use a simple solution. We had to start from scratch. I mean, I wasn't working on that, so maybe I'm not telling super correct things, and I'm sorry, take this with a pinch of salt. But of course, it's a very particular task for a consultancy company. And that shows that we are very, very flexible. We are bold as well, but we are also very flexible. So if you have a problem, we are able to solve it doing the right thing rather than, I don't know, pushing one of our partners, which is the difference between other consulting companies. And I'm not telling that to convince you that NearForm is the best. That's my personal opinion, of course. But for showing people that there are multiple ways of doing consultancy. And it's not easy to tell the difference between consultancy and product if you don't acknowledge that there are a spectrum of, you know, consultancies out there.

Yeah, there's so many different ways.

And of course, when you work— I know a lot of people doing consultancies that says, no, I will never do consultancy again, I just want to go on a product. Okay, all right. But just for you to know that I've been working at Paramount, which is a global company. Basically, we were competing with Netflix, etc. And it was a product company. I was super happy to work there. I worked on the platform team, so I had to develop an internal web framework for our applications, etc. But I've been talking with a lot of people coming from other product companies that say, I'm on a very bad team, on a old legacy code and whatever I do, it's refactoring stuff every single day, which is not good. I would prefer being a consultant so that I like every 3 months I can switch projects and maybe there's something good coming up next. So I would say the difference is not between consultancy and product. It's between one company and another. You know, there are many product companies that has a super cool product on the outside, but when you go see the internals, they are super crazy. And you don't want to work on those. So, um, I know it's a hot topic, but it's really hard to determine which one is the best. It's not about the kind of company, it's about the vision of the company and the internal morale that we can have in that company. That's what makes the difference, not the kind of, uh, business model that we have.

No, no. And I mean, in the same vein, I mean, some companies on the outside, it seems like they're doing something pretty unexciting, it's a boring product, but there are tons of interesting things on the inside. That's definitely it.

I mean, I work in a full-text search engine. This is boring AF. This is truly boring. As a product, it's boring. But the internals, I love them. You know, that's what I guess you were referring to. The fact that sometimes what you're doing, it's hidden. That's fine. Let's just, for example, I go to conferences and show the internals so people can join as an open source contributor. Because that's something that companies need to understand, for example, and is that, again, a lot of contractors or consultants want to go to product companies because they feel like consultancy companies are not doing very cool stuff because of, you know, many reasons we already discussed, which is not true in my opinion. But these are the reasons. But if these companies started to go like ThoughtWorks, for example, They do consultancy, they do that very well, but they also advocate for the stuff they do in conferences, on blog posts, etc. So you know that ThoughtWorks, for example, it's a beautiful company. Why not going there? Neoform is better. That's my opinion. I'm a bit biased.

You're supposed to be. It would be really weird otherwise.

I had to say that. But okay, Neoform, ThoughtWorks, I don't know, McKinsey, for example. There are many different companies that do consultants, but they do that very well, and they show that the culture, the company culture, it's great. And I guess these are the companies that are making a difference in the consultancy world. So one thing that people tend not to understand is that when you go to a conference, for example, as a speaker, I do a lot of conferences. I did like 50 conferences this year. And of course it's expensive for us to send me all over the world, you know, for talking. But I'm showing that our company, it's not that the kind of company that you could imagine if you think of consultancy, for example.

Yeah.

So that's the kind of stuff that people should start doing, in my opinion, of course, to start showing that it can be different.

Yeah, yeah, exactly. I mean, I've heard at least one other speaker pretty much say the same thing about their company. We noticed that we have success in talking about things that we found valuable to others and like telling people about what we do. I mean, it's, it's very true. It's one of those open secrets, I guess.

Yes, exactly.

We really know it somewhere, but we're not—

yeah, the fact that it's super expensive for companies to pay. Like, we, we have a DX team, so developer experience team. We all do open source software exclusively and conference engagements, and super expensive. Of course it is, but it's then easier for us to hire to show our products. So in the long run, it is worth it.

Yeah. And so you just mentioned that you sort of considered open source a third type of organizational company. So what's the major differences or similarities?

Working hours, first of all, you know, but that's my choice. Of course, I work whenever I want. I mean, of course I have to work 8 hours per day, but then maybe at like at midnight I got nothing to do and I'm still awake, so why not looking if there's a new issue? Then I start working on that issue, you know. And open source, it's also full of great people that can mentor you and teach you how to do stuff, but it's also full of people who is not really into the open source mood. So if they find a bug, they say, okay, you should definitely fix this because my career depends on this. You have to do that. You have to do this one thing. So really difficult from time to time to deal with people. And you have to consider that.

Yes, it's a lot of people to deal with.

Yeah, it's a lot of people, exactly. You have to deal with that and you can't expect to change people. You have to be the one to change your approach, your mindset, whatever. And open-source software is basically teaching you this. At least to me, it's teaching how to deal with different cultures, different approaches to software, to life in general, to other people. Because you never know who you're talking with. So, you have to be very aware that a response I could give to an Italian, for example, which is— I'm Italian, so I know how to deal with Italians.

Yeah, of course.

The same response to, I don't know, a person from another country very far from here can be offensive, for example. So this is teaching you to be polite, this is teaching you to take it easy, you know, sometimes. But it's a matter of relationships at the end of the day. So it's not just software. Open source to me, it's relationships with people.

Yeah, yeah. I mean, I follow Donny Osteen, the creator of curl. Oh, okay. I mean, he I mean, he personally gets so much interactions with people. I mean, someone finds the curl copyright notice in their BMW's infotainment system and emails him asking for support on their car and stuff like that.

Yeah, that's crazy.

Yeah, I mean, there's a lot to it that's not code.

Yeah, exactly, exactly. And like, it's also having a vision on a product if you are the creator, for example, or if you're a lead maintainer. You have to embrace the vision for that product. You have to discuss with the leadership for that product. So, it's teaching you that writing code is never just writing code. It's not easy to understand at first.

No.

It's not always about code.

No, exactly. And then it can be sort of a harsh thing the first time people get in touch and maybe they rub you slightly the wrong way or you're in a bad mood or something.

That's what happens.

Yeah, I mean, maintaining so much open source within the company, do you have some kind of structured way of dealing with this, like philosophies around it?

Absolutely. We have an open source policy which I cannot disclose right now. We also have a team that I can— I mean, it's pretty known that we have a team where people not working on clients can do open source software, and we have a brilliant engineering manager working there. Helping organizing all the activities. And the engineer manager works exclusively on that and he's doing an awesome job. We also have the DX team, of course, that is my team that works on other stuff on the open source spectrum. And there is this kind of culture where open source has changed, in my opinion, the way companies like Niaform works. So I'm not saying it's a total priority because at the end of the day you have to pay salaries. And you need to get money. But it's definitely something to keep in strong consideration whatever you do. So sometimes, for example, I find myself having to decide, should I go to a conference or should I dedicate this time working on open-source software? And sometimes I just have to refuse a travel and stay home and code. So this is the kind, for example, decision I have to make from time to time. Because open-source software, it's too important for us to think of giving it less priority than other activities, for example.

Yeah, I was just thinking that sometimes it feels like every single open source project has sort of their own little way of doing things. Yeah, but, and, but then when you, when you maintain so much, there has to be a lot of, you know, what do you say, like advantages of scale, or advantages in coordinating and doing stuff roughly the same way.

Also consider that This is a problem, especially for solo developers. Yeah. Because if you have a company sponsoring your project, it means that if you go, let's say, on holiday somewhere, you're not really necessary. I mean, unless there is a very particular bug and you are the only one knowing that particular part of the software. But if you have a company, you're not in that situation. If I leave tomorrow, people are able to continue working on Lira, for example. Of course, I can do that too, but there is a whole company supporting this. So, it's really important, especially given that we maintain a lot of open-source stuff. Having a company behind that, it's really, really important for that reason.

That's a really good argument for gathering more projects under one, some kind of umbrella, just have more people.

There are a lot of open-source softwares where people tend to avoid them because they are brilliant, they do an awesome job, but there is just one developer.

Yeah.

So what happens if that developer tomorrow stops to write code?

Yeah.

Because it's burned out.

Yeah, exactly.

Yeah, that's a huge problem.

Yeah, I mean, I'm using a project at work that seems to be in that, yeah, in that state.

In JavaScript, we had like a maintainer for a very popular package that went to jail, you know. That's something that happens.

Yeah, you know, of course it can happen.

Yeah, I mean, there's some And how are you expecting to deal with that?

Yeah, that's a huge problem. Yeah, I guess I have to look at the code myself all of a sudden.

Exactly.

So, it's— thinking of npm packages and stuff, it's so easy to build such huge structures of dependencies and stuff like that. Do you have any thoughts around that? I mean, is it something we should, I don't know, try to do something about?

I don't know. I guess it depends on the project you're writing. For example, while writing Lira, my full-text search engine, we had no dependencies. Our choice is don't use dependencies. Otherwise, first of all, you don't learn stuff. And second, we want to support multiple runtimes. So, for example, Deno, right now it supports npm modules. When I started coding the Lira project, it wasn't supporting npm. So, of course, I had to choose, like, Do I want to support multiple runtimes or do I— oh, I don't. Okay, no, I want to. So, no dependency. It's just like the first thing to consider. Also, another argument I have is if you, for example, have to deal with quite complex algorithms, it is okay for me if you want to use a library that is battle-tested and whatever. But while you're prototyping, please do that by hand. Do that from scratch so you can understand why that library works that way. And if you find a bug, you can go to that library and follow everything and maybe also fix that bug in, like, one day instead of waiting 15 days for the author to get free and fix it because that's what can happen, you know. So, about npm, there are, like, if you want a web server, of course, you can't write it from scratch. Or a full-text search engine, you shouldn't write it from scratch, of course.

Who would do such a thing?

Yeah, who would do that? Crazy stuff. But if you want a web server, for example, and you use, let's say, Express or Fastify, which is even better, of course, you have to use it. And you have to get down all the dependencies you need to make it run. And we can argue that the way the Node ecosystem works is not the best one. Of course, I do prefer the Deno way, like you import stuff from a URL, so you don't have to use Node modules, etc. But we are in a situation where we can choose, or we can write everything from scratch every time, or we use it and we start using nice and intelligent approaches to the problem, like Turbopack, for example, or TurboRepo from Vercel. They know that downloading everything from scratch, it's really inconvenient, like traversing the packages on log file or whatever. So, you cache the dependencies. You don't have to install everything from scratch every single time. There are multiple approaches you can take to avoid a problem that we definitely created as developers. But we have to face the fact that the problem exists. We can't avoid that. The only thing we can really do to make everything backward compatible is be smart and try to think out of the box on how to solve these problems. It is not going to be an immediate solution. Maybe, 2 years, 5 years, the Node.js committee will decide to avoid Node modules. We are not there yet, so we have to play smart.

Yeah, yeah. And also, I mean, I tend to think that it's the fact that this is a problem is— yeah, I mean, it's because it's been such a huge success. It never would have become a problem otherwise.

Absolutely.

And I think it's really good what you said about, you know, try to write the first solution Yes, yourself, so you understand the problem. Because I feel like it's, it's so easy to like import a package and imagine that the problem is solved for me without me needing to understand it.

I'm not saying you should do that with React, for example, of course. But like one very popular package, it's— and I love it— it's FastQueue for generating queues. Yeah, of course I want to use that, but I also need to understand how a queuing system works.

Yeah.

So, first, I do a prototype. Then, there is this package built by one of the greatest engineers out there, battle-tested with a lot of issues fixed. Of course, I'll be using that. But I know how it works. And you can't expect to have great knowledge and a 100-foot view of your project if you don't have the knowledge of every single part working, how it's working. Not saying that you as a developer on a giant project, go in every single detail, but you should have people and teams focused on this part of your project, this other part of your project, and here you need people that deeply understand how they're solving the problem. Especially when you talk with data structures and algorithms. Let me give you an example. When using a prefix tree and when using a radix tree, they're pretty similar, but I want you to understand that they are not the same. Why you're using this and why are you using that? Please, when you can answer that question, we can proceed implementing stuff. So, you can implement it, you can understand that one is more optimized than another for a specific use case, then you can use your favorite library out there. But first, I want you to understand why you decided to use that structure and not this other one.

And not just because that one seemed more popular.

Exactly, that's my point.

It's a very good point to remember. Yes. So, one of your talks, my mental schedule is all messed up, so I don't know if it's today or tomorrow, but one of your talks is about refactoring.

Yes, exactly.

Has it been already or is it tomorrow?

No, it's today, actually, at 5 PM, I guess. So, yeah, I have time.

Yeah, then there's still hope.

I have a chance of catching it.

That's a good thing. We'll link it in the show notes, of course, as well. But you can choose if you want to sort of spoil the contents or not.

Oh yeah, why not?

But where to start? We had a discussion somewhere about refactoring, and I started to feel like people mean different things with refactoring. Like, I think it's the original definition to change the structure without changing the interfaces.

It depends on the domain.

Yeah.

I can give you an example if you want.

Yeah, of course.

I would be— okay, that's a spoiler. I'll be talking about that in my talk. So, in Node.js, we have different versions. Right now, the LTS versions are 14, which is coming to an end of life, 16, 18. So, on Node 14, if you have an unhandled promise rejection, it will say, okay, no problem. Just for you to know, starting from 16, so the next major version, this will terminate the Node.js process with a non-zero exit code. Oh, that's bad. So, you know that if you want to upgrade from 14 to 16, which is the next major version— so, always even versions are the major ones— so, if you want to do that, you know that you have to go through your code and you have to refactor. Basically, you have to wrap your promise rejections inside a try-catch, handle the error, do stuff. That's what I think it's refactoring, for example. That's a part of the refactoring job because it's not just wrapping stuff into a try-catch. It's about understanding why it's failing and how to deal with failure. Or another example I will bring to my talk is you have this React component, you want to substitute that React component with this other one, but you can automate this. So, I'm gonna show you how to automate this process, for example. And to do that, spoiler alert, you have to learn trees. So, how to work with abstract syntax trees. So, you know, it's not an easy task. No. But it's an interesting one because when I was working at Paramount, we had very large codebases. And whenever we needed to refactor something real quick, we had to understand how to approach a refactor on a, I don't know, 50,000-file codebase. Yeah, it's not easy, you know. So, my talk is based on my experience at Paramount, which I loved, of course. But yeah, that's really it. It's about changing the program while maintaining it up and running, basically. And while maintaining a release cycle that can be pretty quick and not blocking everyone because you have to refactor. You can approach refactors in multiple different ways.

That sort of explains a lot, I think, because I think my thinking of refactoring was much more narrow. And I realize that whenever someone else talks about refactoring, it seems to include so many more things.

Yeah, maybe also like dead code elimination.

Oh yeah.

That's refactor, possibly. I don't know. Performance enhancements is refactor. For example, in Lira, we have an array intersection function. So you have multiple arrays, you have to find the common elements inside all of the arrays. Right now, it's not really efficient. So we will need to refactor that function. With the same API, you have to provide me a better method for doing that computation. That's another example of refactoring.

Yeah, I mean, this feels like a sort of classic example.

Yeah, exactly.

So roughly how large is the Lyra codebase?

Oh, it's not that bad. It's like, I don't know, would be like 2,000 lines of code. It's not that bad.

No, that's a—

No, it's a very simple thing. Once you understand how full-text search works, It's brilliant. That's why I love it because it feels like, okay, I should have thought of that. You know, it's that kind of feeling. So, it's brilliant. I love it.

Yeah. It sounds like it might be one of those problems where the more you understand, the less code you need for it.

Yes, exactly. At the end of the day, it's really easy. It's about integrating data structures. Yeah. That's really it.

I have way too many React components.

Absolutely, absolutely. Yeah, that's common.

Yeah, too many things to choose from. Uh, yeah, you've written books as well. Yes, or a book, several.

No, no, it's a, it's a paper book. Um, it's about Next.js. At Paramount, I used to work a lot of, on, uh, on the architecture side of Next.js, so not directly on the code, but on how to integrate Next.js in our systems. And it's cool because it went out in February, and like 2 weeks ago, they announced a new version that made my book completely obsolete. Oops. Yeah. So, I will never write a book about a technology that changes that frequently. But writing books is not convenient.

No.

Let me be honest. It will never be convenient, but it opens a lot of doors. So it's really, I mean, the convenience in writing a book is not about earning money because you spend too much time writing books, but people start to recognize you as an expert, let's say, in that field. So they start inviting you to talk to conferences or making, I don't know, private events, whatever. So it's not bad. Also being a published author has its relevance on a resume, of course. So if there's anyone listening wanting to write a book, please You will never be rich. I mean, okay, I wish you to be rich very soon, but it's really unlikely. Yeah, let's say that it's really unlikely.

So what, how long did it take to write?

One year. Yeah, and in one year they released like 3 different versions, so I had to rewrite a lot of stuff. At a certain point I said, okay, this is the book, let's publish now. Don't wait, please.

But did you enjoy the writing process and stuff like that?

No, I gotta be honest, I didn't. Okay, it's, it's really hard, especially for a non-native English speaker. I'm Italian, so writing in English, it's really, really difficult. And like, reviewers will send you back the page saying, okay, you have to edit this, this, this, and that. So it's not easy. If I ever write a book again, I will write on something that it's not gonna change soon. Yeah, like data structures or algorithms. It's a perfect example. Yeah, or like a book on how full-text search works. Yeah, this is the kind of books I, I can see myself writing. Um, because I can take the time I want, because I was in rush, you know, with, um, ever-evolving technology. I was really rushing. So I want to take time, go slow, have, uh, be 100% sure about what I'm writing. So, yeah, that's really it.

So, what is it that you like about Next.js?

Okay. I don't really like front-end web development, got to be honest. And I wrote a book on front-end web development. So, you know what you're talking about. I'm not contradicting myself enough, I guess. I like the way that basically at Paramount, we were writing an alternative to Next.js, but we started writing it before Next.js was a thing. That's the fact. So, we had to maintain it. And I saw how Next was dealing with different problems in a brilliant way. I still believe that the software we wrote, I don't think was better, but well suited for our purpose. Let's say that. One thing I love about Next.js is that generally speaking, it is good for 90% of the use cases while other web frameworks cannot tell the same.

Oh, so it has more flexibility?

Yeah, it's really flexible. But when you really need something like multi-tenancy, for example, you can't do that. Okay. So, that's a huge problem for me, especially at Paramount where we have like 200 websites from a single web server. So, multi-tenant to the core, you know, and I had to hack Next.js the bad way to get it to work on a multi-tenant setting. So, I still like Next.js. If I have to build a website right now, I find myself writing in Next.js eventually. But I don't really like the web development area anymore. I want to be honest. I like to work on other stuff right now.

So, what approaches to solving problems did Next have that you liked in others or that you enjoyed better in Next?

Yeah, it's just easy. That's the number one problem. The average level for developers entering React, we have two types of developers. We have old-school developers that start working in something different and have difficulties in doing modern web development. Next.js makes it easy. That's really it. So, if your level is not good enough, it would be very easy to you to grow up into Node.js. And it's way easier than playing React app, for example. So, it's really convenient. Then we have the new school developers, like the young guys coming here and doing everything very good, very fast, whatever. And they are already pretty productive, but they will be double productive on Next.js because it's easy. Especially for people with a good background or new to the programming world, it would be very easy for them to grasp how it works. That's what I like about Nuxt and other frameworks. Again, we now have Nuxt, for example, from Vue. We have SvelteKit, etc. But when Nuxt.js started, it was the first. We have to appreciate the fact that they invented many things we take for granted nowadays with other frameworks. That's something to consider. It's not. And as always, let's pretend for a moment to be a venture capital wanting to invest. If I were— if I was a venture capital, I would say, okay, it's not just the product, but it's the fact that you were the first doing that. So I'm betting on the team, and I guarantee that you will be creating a lot of new great stuff. So I trust you for that reason. It's easy to copy, it's hard to come up with these ideas. So that's Something I appreciate about Next.

Yeah, I mean, it feels like you could say some of the same things about the React team, like the virtual DOM and stuff. Yeah, and like, let's solve it this way.

Yeah, absolutely. They are from Facebook. Yeah, so, you know, they have a different approach.

Very, very different situation.

While Vercel, uh, for Next.js, they have— it's a different stuff, you know, smaller team, but super, super cool developers that they're really, really good. So, um, that's something also I consider someone that came out from nowhere basically and did a great job. Uh, that's something that I feel like I want to appreciate.

Yeah, of course, of course. So, but, but if you, if you got to choose completely by yourself, you wouldn't be doing so that much front end?

No, I'm not doing front end anymore right now. Um, I work on Lira. That's what I do. Yeah, I like the data— sorry, the computer science part of programming. In fact, when I talk with people, we often say, oh, what is better, front end or back end? It's not— it's— no, they're the same thing to me. I mean, this is web development, then we have computer science. Yeah, I'm not saying one is better than another. I'm saying that— I'm saying my preference when I have to work, when I have to work, not when you have to.

Exactly.

You know, when I can choose, I I choose computer science because it's, to me, that I don't have a degree, etc., to me it's more challenging. Yeah, but it could be easier for you. So if you want to find something more challenging, maybe you find bigger challenges in the web development area. Yeah, so it depends on the kind of person you are. Yeah, I'm the kind of person that likes to be challenged, so I go on the hard path. Yeah, of course, that's really it.

Yeah, and it feels like maybe it's wrong to say that it's underappreciated, but in a way It feels like that because all the excitement is open around web development, frontends.

It's always evolving.

Exactly. But you can make such enormous impact with a better algorithmic solution and stuff like that and thinking about data structures rather than—

Yeah, the fact is that it's really— okay, that's a very personal opinion. But for people born— I was born in 1995. So I grew up with video games, etc. And for people like me, we have a different way of processing information. I'm not saying it's better, again, we have a different way. Like with video games, with already all the information in mind, it's really easy for us to— like, I was born in '95, so the web already existed for 4 years at the time. So when I started using computers, we already had Google, for example. So, it's a different approach. And for people like me that saw many different evolutions in a very short period of time, it's really easy to go on the web development track where everything is evolving so fast because we're already used to that. While if we think about algorithms and data structures, they are never changing. They are always the same because the computer works always the same way. We will have a major breakdown maybe with the quantum computing stuff.

It could be.

It could be. Yeah. But if you're the kind of person that needs slow changes and deep understanding on how stuff works because you couldn't like go on Google and search everything, I can understand that you can have a preference for computer science, for example. I can understand this. And that's really it. I guess we have different types of programmers, but it's also a thing where it also depends where and when you were born, you know, that's affecting the way you approach, uh, this, uh, this job. Yeah, yeah, it's a very personal opinion, of course. I'm not telling truth.

Okay, personal opinions is what we do. Yeah, I mean, I'm not much of an algorithmic person. Okay, myself, but I'm, I mean, I don't— I can't imagine how, like, people writing games and software in the '80s or something got stuff done.

Yeah, it's incredible. They had to write books, you know.

Yes, there was one book and it wasn't in their local library, so they had to order it from a back of a magazine.

No Stack Overflow, no Google. Yeah, it's crazy. But you see, and nowadays, uh, with, uh, for example, when writing, uh, Lira right now, I'm solving a problem of ranking documents. So if you search, for example, quick brown fox, you want to get the documents containing these three terms more frequently. But it's not just about the frequency, but the relevance inside the document. So you use TF-IDF and cosine similarity, which are two different algorithms to determine the relevance of terms inside a document. There is not a lot of— and that's surprising to me. I thought there was more literature online about that. We have many, many blog posts, many YouTube videos, but they are not at the level of detail I was expecting. So I'm finding myself in the situation where I have to buy books, read books, especially from the universities where it's pretty accurate, so I can really understand what's going on and then implement stuff. So it almost feels like being in the '80s.

So there is like academic writings about this?

Yes.

Yeah, but not anything. There's a level missing.

Exactly, there's a level missing. Yeah, there is the super easy stuff and then there is the super hard one. And I feel like the easy stuff, it's not what I need right now. No, it's like good for a shallow understanding of the problem, but you have to go deeper. But when you go deeper, the next step is super hard. Yeah, there is an entire class of tutorials. Yes, you know what happened here, which is an opportunity for someone wanting to write tutorials. Of course, maybe I will. I don't know, still, I still don't know. But, um, it's really, it's really strange to me. I was expecting more literature online on that.

Yeah, and I bet there are, I bet there are many areas like that. I mean, that's also something I've tried to ask people about every now and then. It feels like there should be more bridging we can do between like academia and, yeah, and what do you say, uh, the industry. There has to be a lot of stuff that's stuck on one side.

I would say also for people, I mean, I work with a lot of people who have degrees and they have such a deep level of understanding of stuff that, of course, if they write something, it's 100% accurate and it can't be easier than that.

No.

Because if you want to be deeply accurate, you have to go the hard way, you know. And making stuff accessible It's really, really hard.

Oh, yeah.

It's damn hard. So, you know, I have a talk on how I built Lyra. And it's really hard for me to make people, like, with 2 years of experience on React, understand, for example, how to think in terms of trees and hash maps. And I don't know, Levenshtein distance algorithm, stuff like that, dynamic programming. It's not easy, but it's not their fault. It's my fault that I can't explain that like I'm 5.

Yeah, exactly.

That's my problem.

If I had more time, I could boil it down even more and like think exactly in a short way.

And that's something I'm actually trying to do. I have a talk, I just gave a talk in Berlin called Explaining Distributed Systems Like I'm 5. Because my goal as a speaker, it's becoming to fill that gap in the communication industry, let's call it that way. I don't know how to call that. And I have another talk which is called Math or Magic End-to-End Encryption explained to a 5-year-old. So where I explain, for example, how Diffie-Hellman algorithms for end-to-end encryption works. But I want to do that in a way that if you are a 5-year-old kid, you can still understand it. So no deep math required, no strange formulas, just this is how it works. For example, this is for sisters. I explain them with ice cream. This is how it works. Because if you have one ice cream truck and a lot of people, you can't serve everyone with a single ice cream truck. So you have to take, you know, many, many changes to your approach of selling. Oh yeah. And, um, and that's how I want to explain stuff. And it's super complex. Yeah, that's, that's a major challenge again.

Yeah, exactly.

I mean, but I, I wish it can help people understand why we take certain decisions when we want to solve complex problems. Yeah. So that's the ultimate goal, to bring people that's interested in this to a level of understanding where they can go on the hard path with academic papers, or they can stay there and start to do stuff, basically.

Yeah, yeah, apply it. So yeah, we covered a lot of ground. I guess it was exactly what I was hoping for. Maybe we should start rounding off. Is there anything else you think I should have asked about?

No, one very last thing. I will do a talk tomorrow also at Arda. About all the programming languages that are not in English. Oh yeah, I know that can be interesting.

Yeah, this is for me.

Yeah, because I'm a super programming language nerd. I love experimenting with different programming languages and paradigms, and I find this to be a very interesting topic, and it brings you back to the history of programming itself to understand why we are all coding in English. I mean, I don't know where you're from.

I'm from Sweden.

Oh, from Sweden. Okay, not an empty language speaker. You got a good accent, actually.

I wasn't expecting that.

I know Northern European people speak very well in English. That's very true, I guess. It's a good stereotype.

Yeah, let's go for that.

But you can tell by my funny accent that I'm not a native English speaker. So, I find it attractive to understand why should I have to write Python as if it was English language. Why?

That's pretty weird.

Yeah, exactly.

And there have been so It seems to me at least so few attempts at doing anything differently. I mean, I know at our company years back they used to, you know, one of those fourth generation products that was called, I think it was just called Fourth Dimension, 4D, and that was actually a French creation. Oh, okay. So in that one you actually could switch languages and all the keywords would switch too.

Yeah. Okay.

So that's sort of one step in that direction.

I guess they did similar thing with BASIC. You can translate it to whatever you want, basically, because it's so easy. It's pretty easy to do. But there are also languages that have keywords, but you can write code without them. Like Prolog, for example. You can write Prolog in English, or you just avoid using keywords. Or, okay, if you go very deep into the rabbit hole with certain Lisp dialects, you can write everything without any keyword at all. So, it's mad. But, of course, this is not the point of my talk. I want to talk about languages that doesn't use text, for example, at all. Yeah. So there is Velato, for example. This is the only spoiler I'm going to give you. There is Velato, which is a score, basically. And if you play like a piano, a MIDI piano, and you play that score, you make an Hello World program, which is crazy.

Wow.

But you see, I love the fact that people, it's so creative when it comes to programming. And that's what passionate me about that industry. Because you can literally do whatever you want. You are like Archimedes, you know, you can invent stuff. You can be an inventor when you're a programmer. And this talk specifically, it's about that, being creative and do something different.

Yeah, sounds great. We're linking that as well, of course. Thank you so much for taking the time.

Thank you for having me. It was a pleasure.

Yeah, it was great.

Let me put it this way, Mr. Aimer. The 9000 series is the most reliable computer ever made. No 9000 computer has ever made a mistake or distorted information. We are all, by any practical definition of the words, foolproof and incapable of error.