Dhyey Mavani: From Triple Major to Model Whisperer

Hey there and welcome to the userexperience.org podcast where I chat with the people creating the products that shape the world. Sit back, relax, and enjoy the show. All right, so we're live. Welcome back to the userexperience.org podcast where I interview people working in tech to hear their thoughts on the future of the intersection between humans and technology. Today on our show we have DA, and, um, yeah, first of all, welcome.

Thank you for having me here.

Yeah, you're welcome. And I think how we ended up meeting or connecting was at the AI Startup School at Y Combinator. I don't recall meeting you in person, so I'm assuming you saw the poster and then scanned the QR code.

Yes.

Nice. So I always like the people that scan the QR codes because I, I can like filter out people like that way. I think people who scan QR codes are just naturally curious and interested in like learning new things. And I think looking at your LinkedIn and just talking to you briefly, that's like a theme that really resonates with you is you're really curious and you like learning and you also just like, you know, putting yourself out there and achieving really cool stuff. I'm gonna try to really briefly like say your bio and then I'd like you also to introduce yourself. But basically DA is a triple major in computer science, mathematics, and statistics from Amherst College, which I think is in, like right outside of Boston, right? Uh, yeah. So I actually went to UMass Boston. Cool. Yeah. And, yeah, you've worked at, uh, you're, I think you're working at LinkedIn or you have worked at LinkedIn, also at Amazon and Valkyrie. And you have this just incredible resume. And you also like doing some crazy stuff like jet skiing and quad biking, bungee jumping, and ziplining. So yeah, super excited to have you on the show. And, uh, yeah. Could you just tell us a little bit about who you are and what you're working on and maybe a little bit about your origin story?

Yeah, for sure. I'm from Junagadh, Gujarat, India, and I came to the US when I was 17 to pursue bachelor's degree, which Oliver said, in computer science, math, and statistics. Recently I wrote two theses, one in statistics and one in mathematics, both focusing on applied computational mathematics. And previously I had the experience in applying computational mathematics to a field in agrivoltaics. Over the course of my time at Amherst College, I also built a couple like edtech-ish tools. One of them is called Artblocks, which I can talk more about and we can chat more about in the podcast. And as Olivier said, I interned at Amazon and LinkedIn, and I'm returning to LinkedIn to pursue a career in software engineering, focusing on generative AI.

Yeah.

Awesome.

So then if we could quickly take a step back to when you moved to the US, how was that for you?

Yeah, I think it was a great experience because I felt everyone was really welcoming and pursuing something that's like not so accessible here in my country, like a liberal arts education, focusing on the stuff that I like especially. It was really helpful because Amherst had this open curriculum, which was really amazing and allowed me to take a bunch of courses that I always wanted to take and without worrying about the core requirements. Apart from that, I got a lot of opportunities, which was amazing, outside the academics. So every summer, winter, or even during the school year, I was able to kind of work through a couple of exciting projects that I always wanted to work on after getting excited about certain topics in my courses or outside of them as well. And having a great friends network and also having great, like, colleagues really helped me. Throughout my 4 years here in the United States so far.

Yeah. Awesome. And so you mentioned that you've already started making some projects there and they became quite successful. Could you tell me a little bit about more about them and also, I guess, how that relates to the ultimate goal of those projects?

Yeah, for sure. So I can talk about the first one. I think it started my freshman year. I think the spring semester of my freshman year. So we noticed that— me and my professor noticed that there is Google's Blockly a software that empowers the entire stack of Scratch, scratch.mit.edu, which is like a block-based programming language. And it had support for Java, C++, and Python and all, but it didn't have support for R, which is a commonly used statistical programming language. So we noticed that and we're like, why not? And we needed it for, so we needed, like we had some beginners who had trouble learning R because of course they're learning it for the first time. And it's a different way of thinking. So we were like, okay, how can we support these people? And block-based programming had success in helping people learn programming, especially ones that are beginners. So we were like, okay, let's build something that uses Blockly, but then provides an interface with R so that people can learn it. So we built RBlocks, which is a web application. I think it's available online. Now it's deployed and we're working on like publishing a paper on it as well in the next 6 months. So the whole idea is like it's block-based. So you like, you can drag and drop blocks. It gets converted to R code on the side in the compiler, and then you can hit compile and everything works. So the idea is to not let people get discouraged by the error messages, pesky error messages that you get when you first start writing lines of code. And you have like this nice interface where you can just do a right-click and have like a help panel. You can have documentation get pulled up when you just click like a documentation button there. Collapse the blocks, expand the blocks. You can add parameters within the blocks so that it's kind of creating this meta-level schema in your brain on how to program when you're doing it for the first time. And it was deployed— so the earlier version was deployed through the IT department at Amherst College, which we are really, really grateful for, especially Dr. Andy Anderson. And that was used in one of the intro stats courses and reported huge successes. I heard a lot about, that tool in conversations at our Valentine dining hall, and people came up to me and like, uh, said that they really enjoyed using it. And we got a lot of like reviews as well. So we are thinking about upgrading it and changing some things before we make it into a paper and before we release it to a broader audience. So we deployed it on now a separate web application that's independent of the IT department so we can have more control and a more fast pace on it, a more fast-paced iterative development on it. So that's that. And yeah, so that's the entire timeline of how that happened.

Yeah. Awesome. Well, congrats on that. Last question about this. So you mentioned you're going to change some stuff as well. Like, were there any things that surprised you where you thought, oh, this should be really easy to use, or like, we think this makes sense, and then you got some feedback that actually your mental model of how the user's mental model was weren't necessarily aligned.

Yeah, I think one interesting thing was kind of the boilerplate blocks. We thought that users would be familiar with the for loop kind of functionalities or kind of embedding blocks kind of functionalities where you can have a block with certain statement up top and certain statement at the bottom, and then you can add the blocks in between, but it was not quite intuitive how to do that. So we had to realign the interface to be more explicit about that and make some space there. So it's more intuitive on the user experience end that, okay, there is something that can be added here. And there is this way that I can also program my blocks to get converted to R code, which can make it successful. So essentially making sure that the assignments have multiple answers possible on the blocks end and just making the compiler better because earlier it was refreshing every time. And we wanted to make sure that the user— users are able to like save blocks and save their progress automatically, which which wasn't quite evident to them. So a couple tweaks were on that front as well.

Awesome. Cool. Yeah, I'm just kind of taking a look at the other stuff that you've been doing. I mean, I have to say I've interviewed, I think at this point at least like 25 people into like long form interviews. I still have to edit and post them and no one has had such a like crazy LinkedIn that I'm like scrolling and scrolling and scrolling and It's just a, it's a lot, which is really cool. So congrats on that. It just makes my job a little bit harder where I have to like, you know, find the things that I choose what I want to talk about. Um, but yeah, I'm just kind of scrolling. I see here, you know, software engineer at AWS. I see here like a dashboard. Um, are you allowed to talk about that? Yeah.

Uh, but I think what would be really helpful is probably talking about the, the, uh, the true thesis. And I think they kind of align a little bit better towards the user experience front. And I think those are the most, more recent experiences tying in.

Yep, sure. Let's go.

Tell me about it. Sure. So, uh, the first thesis in mathematics. So the idea was, uh, there is this new programming language called Lean4 developed by Microsoft researchers. And the whole idea behind the language is if you write a proof in that language, a mathematical proof in that language and it compiles without errors, then the proof is correct given the axioms are correct. And the axioms are like very basic axioms of mathematics that people have already formalized for you. So there's a library called mathlib that has a lot of these axioms and a lot of the basic theorems and lemmas also formalized for you. And people, whenever they formalize a huge chunk of mathematics or like a topic in mathematics, they add this to the math library so people can use it. So it's kind of like a reusable software.

Could you, uh, for the non-technical or maybe non-coders out there, just give a brief, like, you know, concrete example of, like, maybe— I'm not sure, but maybe, like, this is an example in C++ or Python, and this would be how it would go in Lean4, just so we can kind of understand the difference?

Yeah, sure. So Lean4 is a functional programming language, uh, unlike the other programming languages that we are familiar with, like Python. So what that means is like you can pass in states. So let's say you have a problem, uh, 3 1 1 5. That's like you need to prove this, let's say, right? Now in the basic like mathlib, 1 1 2 is defined, let's say, as a lemma, right? So what you can do in Lean is you can say, okay, I invoke this lemma A on my current problem, and that gives me— with a sub— that, that leads me to a subproblem: 3 2 5, right? Now I still need to prove this statement, but now that's a state that's much— that's like slightly better than the state I was earlier, right? Because now I use this lemma, and now I can go off this state and try to like equate the left-hand side and right-hand side. So somehow in order to like make my, make my lemma, make my statement compile. So what we want at the end is 5 equals 5, which is a reflexive property because again, both are numbers, integers, so to speak, or you can also say they are like natural numbers. Those are also lemmas. And then you can say, okay, now at the end, it's reflexive, which means RFL, which is a tactic that someone has implemented for you. So you can just say, okay, now RFL, and that satisfies all the goals. So the goal was proving 3 1 1 5 is now satisfied. So that's one brief example on how to do it. And let's say you have a Pythagoras theorem, right? Like then you can use like a bunch of lemmas that are developed by other people, and you can also like develop your own lemmas and then say, okay, now I don't want to worry about this whole big problem right now. I just focus on this lemma.

Go on an excursion and then come back in. Okay. I hope that my confused face didn't throw you off too much. I think I understand, like, in general, high level, what it is. It's like a programming, programming language where you use, like, lemmas, which are kind of not necessarily assumptions, but, like, building blocks, and then they kind of lead to a proof or something. Um, yeah, I did one, like, I think it was like an intro to logic course where we had, like, axioms and, like, propositions and like if A then B and like not A not B, stuff like that. I enjoyed it, but it was pretty tricky. Yeah. And then, so tell me, so you're working on that, on a thesis about that and yeah.

Like what's, what's the working title? Just so I can kind of understand briefly.

Yeah.

So it's using Lean4 to formalize a problem called chip firing. So just to give you a brief on chip firing, what it is, consider a graph with multiple edges. There can be multiple edges between two vertices. And then let's label each of the vertices as like Alice, Bob, Charlie, whatever. And then each of the, each of the vertices have wealth attached to it. Let's say Alice has $4, but Bob is in debt of $2, which is minus 2, right? And the like. And the goal of this entire chip-firing game, so to speak, is you can lend from each of the vertex in equitable manner. So for instance, if Alice is connected to Bob with one edge,, but Alice is connected to Charlie with 2 edges. If Alice decides to lend, then it has to lend, then, then it has to lend like $2 to Charlie and $1 to Bob at a time. Right? So that's one, that's one lending move or firing move, you can say, right? So after us, so the problem statement is after a set of these potential firing moves, somehow, is there a way for us to get a configuration where no one is in debt? So that's the entire problem statement, and that's called determination of like winnability. So we call a game winnable or a configuration winnable if this is possible. So there is a whole like theory and group theoretic way developed for this, um, and I studied that theory, wrote an exposition on it, um, and there is a theorem, beautiful theorem called Riemann-Roch for graphs, uh, that has applications to algebraic geometry. And how this ties into Applied Computational Math is we built two things. So we formalized this Chip Firing for the first time that we know of in Lean4, which is really cool for people because they can now do more explorations of their mathematical curiosities by just tweaking around like what stuff we have already built, like for firing moves and the like. And the other thing is building a Python package called Chip Firing. So we noticed that there is no single user interface for researchers to execute these moves. So let's say the mathematician has an intuition that, okay, if we do this sort of, sort of like strategy, then it would require less number of moves. But then there's like no standardized Python API or like easy API for them to like try these things out. Right. So we noticed this problem and we are currently working on this Python package called Chip Firing, which is live on PyPI, which is the Python index. And we are planning to publish this paper like soon in 1 to 2 months. So that will be out soon. You can check that out. But the entire idea behind this is again, like building something that allows them to do combinatorics, allows them to visualize, which is a key component because I need to see like how the chips are moving. If I'm a mathematician, I want to see like, okay, if I have this kind of like intuition of my lemma, how does that play out? Or on the other hand, if I know that playing out in this way would be most optimal in terms of the firing moves, then how does that translate to the mathematics behind it? So there is an entire representation of every firing move into like an equation, of course, and some mathematicians prefer to think in equations, and sometimes it's easier to think in graphs, right? Or it kind of like the sets of graphs, right? So if you have like multiple people fighting at the same time, right? In the graph.

So, so that's the idea behind both of the, both of the software components. Yeah, that's awesome. I was actually listening to a podcast, I think less than 5 days ago about graph theory and search theory. I think it was like, it was like, it was a pretty basic one, but they were talking about how like in the 1700s, there was like a place with bridges. I think it was like in Italy or something, and they were trying to figure out, can you like cross a certain amount of bridges like only once or something like that? And yeah, I was just really fascinated by that. Also, what you're saying about some people really like to see things visually, I think that's definitely something that we're gravitating towards. And, you know, always with software and complicated things, like, sure, you see the code and you see the equations, but a lot of times it clicks much better and you kind of get that intuition when you see things visually. So I guess my question to you is like, what key insights can you extract from what you're doing now, what you've learned from doing this to like human psychology and how that kind of relates to software and technology? And then I guess to user experience.

It's a long-winded question, but I think, I hope you get it, otherwise I'm happy to rephrase it. Yeah, I know that's a really interesting question. I mean, I've always been debating how to, better approach building such software. And I think what it boils down to is just talking to your users. So my advisor has been like really helpful to me because he is also working in this area as one of his fields. His name is Professor Nathan Fleeger. Like talking to a bunch of like people who are working in this field, for example, Professor Ralph Morrison from Williams College as well, I realized that there are certain problems, for instance, like gonality and figuring out the solutions to elliptic curves that are harder to figure out when you are just dealing with it in terms of equations. And that's why this entire theory was developed in the first place, or even like thought of. So I kind of tried to base my decisions on user experience from that Origin Insight, because that definitely like turned out well for me, especially for the project as well, because it does save a lot of time and kind of gives you the creator sense of like why the stuff is created and you can align yourself with it. And your broader question of how this ties in this and Artblocks, maybe like ties into the like. Application of applied computational math to the broader, like, human-centered tech development is they're just a different set of users. I mean, everyone has different, like, things that they're optimizing for and different styles, like, from their, like, experiences of how they look at technology as augmenting or just an end in itself or just using it for a certain use case, right? And then rest of it can't be automated right now. Right? So you just want to talk to your users and figure out what is the right balance there and how they are planning to use the technology and where they're getting stuck. What has helped me a lot is definitely like shipping fast because I realized when I ship fast and like make mistakes, then I get critical feedback and real feedback from the users.

And then I can iterate quickly.

I think users from— do you have an example of that? Yeah, yeah, I think for definitely from Artblocks, right? Like, like if you talk to the students, like sometimes they're also like starting off in the software and they might not be able to articulate what they're facing. But if you observe their like click-throughs and how they're approaching to use the software, then I think it's much more evident. So what is really helpful is you you interview the users, but then you also have a critical eye on how they use it. And that gives you a better sense on what to improve for and where your software is getting stuck. And what is the loophole where there is a lot of time spent by users where they're not able to figure stuff out and the cursor movement is just like here and there and they're not able to like do anything with the software that you provided them.

Yeah. Have you ever had instances or cases where like you had to not follow that blindly where people were behaving a certain way and then maybe the initial assumption was like, oh, we have to change this so that this happens. But then maybe if you took a step back, you realized, oh, actually my whole paradigm is wrong. I'm not sure if I'm explaining myself correctly, but my question is, how do you find that balance between using your own intuition and vision and just kind of looking at what they're doing?

Yeah, I think What really helped me in that sense is A/B testing, kind of selecting two different group of, sorry, two representative group of users and then shipping like different versions of software to both of them in alternating those weeks. So we can get like an insight on what's working and what's not working. And generally, if like considering my sample size were not that big, but in the future when I work with softwares that have big AB testing groups, then I think it's pretty objective on what's working and what's not working. From my understanding, I feel like your initial intuition should be based on pedagogy. So I read a lot of papers who are like the— from the authors of Blockly and the authors of like Google Research and the like. And I saw how they structured their APIs and how Scratch was designed. Intricately, and then I try to like take insights from there. But since my user base is like very limited right now, I try to like optimize for their use case for now. And then kind of like having that as a starting point definitely helps like pedagogically and then kind of iterating through and kind of making your user base happy is the paramount is what I think. Yeah.

Awesome. So I see that we're nearing the end time. I did have one more question I wanted to ask, and I also am happy to receive questions from you. But my last question is, so AI is a thing, it's happening. How do you see AI changing how we design software? And then also, how should we be designing software for AI systems? And I guess if you could tie that into what you're doing, especially with Lean 4, because I think what's really cool about Lean 4 is the fact that it works with assumptions or with proofs and then you can truly prove, okay, this actually works or this won't fail. Whereas nowadays we have AI that hallucinates a lot and you don't want to put your whole tech stack into an LLM and then just hope that it never fails.

So I think that, yeah, I asked like 3 questions in 1, but I'm sure you can manage your way. Yeah, for sure. That's a great question. I mean, AI, has like a huge impact in both the software development space where you like develop the softwares and also the user base, because sometimes your users are also AI agents, right? So what you want to do is kind of like strike the right balance there, not just like leave everything to AI as a software developer, being still critical about the user experience and like thinking critically about like the pedagogy is really helpful, but then also having documentation that's readable by AI agents. Like the AI models. So you need to make your documentation AI-friendly, make it into bullet points, tailor it so that your descriptions are like more elaborate for the functions that you're writing. So if an AI agent comes to scrape your source code, scrape the source code of your package, then it's able to better understand what to do. So if a new researcher is trying to onboard using Claude, for instance, from Anthropic, then they are better able to quickly write up the examples for their own use case without us having to say, okay, there is this examples folder, go to the examples folder, and there are these set of examples that we had to craft for you. So if we make it accessible to AIs, then that's also really helpful. And on the other hand, there have been a lot of like research studies on using chain of thought and like large language models of just the plain language models and also like chain of thought reasoning to enhance the ability of models to prove. Theorems and solve mathematical problems. Because if you have like this structured environment, think of it like if someone is checking if the proof is correct or the problem that you have solved so far is correct, then hallucination is limited, right? So it's essentially saying that LLMs generate something and there is this validator, let's say Lean4Software, that's able to validate if it's a hallucination or not. And it's able to give you the errors as well, like give the errors back to the LLM and say, okay, these are the errors, and this is not how you phrase your statement. You should phrase it this way. This is how the API is working, right? And that way, if you do those loops, then you're able to like prove pretty impressive theorems. I read a paper called it like, I think it's titled MA LOT. I definitely cited it in my thesis as well. You can check that out. And that's able to kind of prove, I think, Pythagoras' theorem. And the accuracy there was like 30%, 30, 40%. Up from 9% that was there like a year ago. So it's pretty insane improvement that they're able to get using Lean4, especially with this kind of like GAN-like model, which is like similar to like— GAN means like generative adversarial networks, so which means there's a generator and then there is an adversary that's able to tell you if it's correct or not. Awesome. Cool.

So I think then we're going to wrap it up unless—

do you have any questions for me or anything else you'd like to mention or talk about? Yeah, I mean, I wanted to ask you about your experience just coming from like so many countries and kind of having that spread background and how people like think about user experience and what do you think about how scientific computing or like applied computational math can like progress further with AI.

Yeah, so to your first question about I have been in a lot of countries and I grew up in a lot of countries. And I think for me that just kind of came natural where I realized I lived in a certain culture and then I moved to another culture and I lived within a bubble within a culture and everyone has just a different mental model. And that's something that I kind of realized early on. And I think it also shaped the way I view building software where, for example, at the beginning I would ship products and be like, okay, this is how people should use it and this is how people should view it. And then I just realized right away, like, people are not the same as me and people just have their own mental models, which is like valid and fair and like there's nothing wrong with them. So yeah, I think that for me, that's kind of the way I view everything is through the like lens of mental models. And I think it's also really interesting what you mentioned about AI agents, that they are also users and we also have to think about, you know, how do we build for them? Yeah. And how, and just in general, how do we build for different people? And then to your question about like scientific computing and applied computational stuff. So I'm not very into that field right now. I don't know much about it. I think based on what I do know, which if I— correct me if I'm wrong— is like, you know, at least what you're doing with Lean4, how do you like, yeah, take something that's very abstract or like is prone to hallucinations and how can you use things like Lean4 to make sure that there are no hallucinations? That's something I'm very into. Also back in the Netherlands, I co-ran and co-organized an AI safety group where we just meet up weekly and talk about news and AI safety. And the idea that maybe at some point— this is becoming less controversial, but at some point there might be an AGI or something that might not be aligned with humans. And I think computational mathematics, or however you want to call it, I think can definitely really play a role in that. And if we can prove mathematically or logically or statistically well, I think logically that an AI system is aligned and will continue to be aligned gives us much more confidence to be like, all right, let's go full steam ahead. So I think having that brake or having the steering wheel basically in the car lets you have that confidence to press on the gas. So yeah, that's kind of my take on it. That's a nice way to put it. Nice. Yeah.

All right.

Yeah. And if people want to like collaborate with you or check out what you're doing, what would be like the best way to get in touch with you?

Yeah, for sure. I've listed my emails on my LinkedIn profile and you can also visit diamani.com, which is literally my name.com. And yeah, I'm happy to collaborate and even like talk more about different stuff that I'm doing and I will be doing and just like chat more about what you're doing as well. Yeah, awesome.

Cool. Then, yeah, I'm going to click end recording.

Thanks a lot for coming on the show and let me know once you're back in Sunnyvale. I'm happy to grab a coffee. Yeah, for sure.

It will be great to meet and thanks for having me on the show as well. Yeah, so I'm going to click on stop. Well, that wraps up this episode. I hope you liked it. If you did, please leave a review and share the podcast. I'd love to hear what you thought. If you or someone you know would like to be a guest, send an email to podcast@userexperience.org.

And for more resources, make sure to check out userexperience.org.