Max Sklar: Full Interview

Alexa, play Machine Yearning.

Here is the Local Maximum podcast by Max Sklar.

You have good taste, but I want Machine Yearning.

Sorry, but he brings me ice cream.

So that's how this works?

R-X-I-X-E.

It's Machine Yearning from Assist. Another week where we continue on this adventure where marketers, brands, and entrepreneurs get to have a place to think, dream, and ask questions about the future of AI, the talking internet, and how we're reshaping our culture.

Machine learning.

This week is a rocket. and the rocket has a name, Max Sklar. Max is a machine learning engineer at Foursquare and is a fountain of smart, concise thinking on privacy, social media, and whether we're looking at an imminent AI winter. Shane Mac from Assist got so much goodness from his fellow podcaster. We think this is an episode so packed with goodness, you'll want to give it a few listens and definitely share it with friends. Then, Make sure you check out MaxisPod, the local maximum. It's awesome.

Tell us about your work at Foursquare.

I had worked on a sort of a local search type website back in 2005. It was called stickymap.com. It's still up sometimes.

Sticky map?

Yeah, sticky map. It's still up. It's, it was, this was like 2006. It was like Wikipedia for maps kind of a thing. And people would post little, icons all over the map. I guess you would call them emojis now, and they would leave little messages. And I thought it was really cool that, like, okay, I had no idea how to turn this into a business, I had no idea what to do with it, but it was pretty awesome that people were actually using this thing I built and annotating a map, essentially. You know, "Share your neighborhood" was kind of the tagline that I put on the site. And so I discovered Foursquare in 2010, you know, when I first got my smartphone. But I opened it up and I immediately recognized, oh, this is like a whole other level to what I was looking at a few years ago, because now people are interacting with the different venues while they're there. You know, this is the mobile era now, and, you know, people are checking into venues, telling their friends they're there, they're annotating it while they were there. I mean, with my website, I would go out all day, I would come back at night, and, you know, fill out what I did. Like, that's kind of a drag. And I'm like, wow, this is This is the way to really get it to take off. So I was immediately interested in it, and I was interested in how they wanted to build a really good recommendation engine, and they wanted to use real machine learning. So I was like, okay, this is perfect. This is what I want to work on.

When you saw the check-in, did you think, ah, that's the annotation on the map?

Yeah, yeah, exactly. Like, I was like, oh, and the fact that it was gamified very heavily back then, I was like, because I had trouble, You know, I was like, okay, this is fun for like a small group of people, but how do you get a larger group of people to sort of annotate the map was always kind of elusive to me. It was like, you know, you try to build a good user interface. There are a lot of very difficult problems to solve. Just mobile in general, which seems too obvious now, but in 2010, I guess it was less obvious. It was more mind-blowing, was just the answer to a lot of these questions.

And now that's evolved into MarsBot. What is MarsBot?

Mars Bot was a little app, a little chatbot, I guess, that we built a couple years ago. And that's kind of our stake in the ground as a company as to, okay, what do recommender systems look like, you know, in 10 years from now? And so we sort of tried to reimagine everything that we worked on. Okay, we have all this context, sometimes we call it context bot, not a chatbot, because you don't really chat with it as much, you can, but mostly it's like, okay, we know when you walk into a restaurant or a café, we know what the best things to order there are, we know what the best thing is to do next given what you're doing now. So given that we have all this contextual information, all these contextual recommendations to give, how can we kind of package that into something that's really lightweight? Essentially, you download the app, the app just contains our technology. Which detects your stops. It doesn't really do very much. And then all you do is, you know, you get a phone number and you can receive texts from it and text it back. And so it's very lightweight and it's like, okay, you know, what can we do with that? How can we use all the technology that we had built up to 2015, 2016 and kind of bake it into an app like this? And so it was really, it was a really interesting project to get started on. We still learn a lot by looking at it. It's not under like active development right now, unfortunately, but we may be able to pick it back up in the future.

One thing you said there is the first time I've heard someone say it, context bot versus chatbot.

Okay.

That's a very interesting difference. And I think is what you're saying, it's about anticipation versus automation. It's about anticipating what I'm going to do next and delivering that to me the right moment, the right time, not necessarily me chatting with this bot. The way you meant context bot was that. It's always going to deliver me in the lightest possible way via text what I should do and what I should do next.

Okay. Yeah. Let me give you an example on that. You know, I was in San Diego a few months ago and I was staying with my cousins and my cousin Jordan drove me to the airport. And before we got to the airport, we stopped at a cafe that he likes and, you know, for breakfast. And so we sat down and the waiter was about to come over and all of a sudden I got a text And I looked down and the text was from MarsBot and it said, "The best thing to get here is the English muffin." And I was like, "Okay, well, that's interesting. I don't know if that's what I'm going to get right now." So I went to order and then the waiter said to me, "Okay, do you want toast with that or do you want an English muffin?" And I was like, "Oh my God, well, MarsBot just told me the answer to that question." So of course I ordered the English muffin. Of course it was delicious. And it was like, it knows. And the reason why it knew about the English muffin was because So many people who go to that place mention the English muffin when they check in, they talk about English muffin in their tips. We have all this statistical code and all this natural language processing code that pulls out what we call the tastes, the noun phrases. I mean, I worked on every layer of this. And then, so that sort of endpoint was— and then not to mention the idea that, okay, it knows that I'm in this café and then it knows to give me this recommendation like 2 minutes after I get there. It's like all of that comes together to give you the right piece of information at the right time. And so that's what we mean by context. In our case, context is where you are and a little bit about that place. That's sort of what Foursquare is, you know, really good at. But other services, I would imagine if you want to zoom out to ask, you know, what's a context bot in general, certain services would have to have a certain piece of information that's more than like just the general piece of information, certain information about you, certain information about about what situation you find yourself in right now to really help you out.

Fascinating. This reminds me, I read a quote that you wrote, that you said, and I thought it was a great quote. You said, "The most interesting real-world problems are ones with endless possibilities where it is not feasible to explore the space of all possible solutions. This can be frustrating when you're in problem-solving mode as an engineer, but I've grown to embrace it." Tell me how you came to embrace it.

Yeah, I mean, that's, I guess, what I came to the realization on when I was in grad school. I don't think I would have been able to put it that succinctly at the time, but that's sort of why I was kind of unhappy, I guess, with some of my earlier engineering roles, you know, back when I was kind of new at it my first few years, where the problem was kind of really set in stone. It was like, take these numbers and then combine them, and we already know what the answer is. We just need it automated. And I kind of came to the conclusion later on, no, the interesting problems are— problems that can be automated like that is something that can be solved very widely. It's very easy to copy. But if there's a problem like, you know, giving someone a recommendation or even something as diverse as like optimizing an ad campaign or anything along that nature where there's way too many dimensions and like way too many possibilities for anyone to have, like, the right answer. Well, if you have a product that really figures that out, you're gonna be ahead of the curve, and it's gonna be very difficult for someone to copy you. But also, it's gonna be really, really interesting trying to solve that problem because you're constantly tinkering. You don't really know exactly where you're gonna go with it. Um, sometimes you try to automate your tinkering, but then that automation is something you have to tinker with. So Yeah, that was my realization.

Machine learning.

Told you Max was kind of a rocket.

Machine learning.

Up next, Max and Shane are going to help you sound smarter about the problems of social media. The objectivity-subjectivity conundrum, language classifiers, tweaks to the recommender system. Max hones in on several of the Gordian knots that keep us stuck. All of this has significant ramifications for everyone who is trying to figure out how to blend sentiment, community satisfaction, and technology. Let's get to it.

I want to talk about accuracy, fairness, and gaming the system.

Yeah.

And you've spent time thinking about how we would fix Facebook and Twitter.

Yeah.

They're flooded with bots. They're wreaking havoc on our culture, our tech, our democracy, everything.

Yeah.

Walk us through the solutions.

Okay, let me start.

How do we balance the technological and human solutions?

Yeah, let me start with the sort of my description of the problems, and then we'll get to the solutions. So I look at a problem that, you know, we face at Foursquare. You've faced a bigger sense in a place like Twitter or Facebook, and that's spam, right? And at least with the types of spam that we tend to see, you don't really have this subjective aspect of it. Like, a reasonable person would agree that that's spam and that we need to get rid of it. There are gray areas, but in large parts, you know, we can kind of create a set of rules like, this is allowed, this is not allowed, and everyone agrees. So you build machine learning model, get rid of those. Spammers try to get around your machine learning model, you continue to fight them. So long as you continue to play in this arms race, you can kind of get ahead of them because you can spend more than they're getting out of it. So they're not gonna spend more. So that's one problem that I think is solvable. But now we're facing another problem. There's this law that like eventually satire and like real life crazy people don't sound exactly the same. And then we have a whole bunch of subjectivity in terms of what belongs on a system like Twitter and what doesn't belong on a system like Twitter. and so that's creating a lot of conflict, and, you know, there's a lot of subjectivity in terms of like what's hate speech, what's not hate speech. If you turn it around and, you know, make it, you know, what's something that upsets someone else, well, that's of course going to be subjective, and so now you have an even bigger problem, because not only is the problem adversarial, but people can't agree on what the right answer is to it, and that's a problem that Twitter and Facebook are -facing right now. The reason why it's so hard is because of how subjective it is, and I don't think they're going to be able to solve it in the way they think they're going to be able to solve it, which is to use language classifiers.

What is a language classifier?

Language classifier is like, is this text spam or not? Is this text positive sentiment or negative sentiment? We do that at Foursquare. That is subjective, you know, in a way, but we can get it to good enough where it powers our ratings. So it's like, for some forms of hate speech, you know, the most egregious forms, Absolutely, you could build text classifiers, you can get them off of there. But for kind of the more subtle things, and you also have this idea that people are going to be flagging things they don't like, and then you're going to have to kind of adjudicate around the edges a lot more than we're used to doing. This approach where you kind of assume there's an objective answer is not going to work. And I think I would like to see an approach, and some of the quotes that I've seen from Twitter spokespeople in news articles, some of it confirmed this and some of this say, no, they're not going to do this way at all. It's just to say, you know what, this is a subjective thing and we're going to kind of have an algorithm that sort of tailors our recommender system, I guess, for each individual person. So what's good for me to see might not be what's good for you to see. What I might think is sort of good and proper might not be what, what you think is good and proper. And If we sort of take an approach where it's instead of the input is text and the output is good or bad, it's the input is text and the user who's going to see it and the output is good or bad, then I think they'll keep more people happy and there'll be less kind of contention over what they're doing.

Do you think it's more of a labeling issue? Take really extreme content of car crashes or people dying. Yeah, really bad stuff on the internet that's always had a warning like, hey, what you're about to see is really bad. Here's what's going to happen. Is that the a solution for this where it's more of a free speech warning across, hey, this is a very different viewpoint than it looks like you're interested in, and here's the things to be aware of, and don't watch it?

Well, I think the key thing that you said is this is a very different viewpoint from what you're interested in. So that immediately just makes it, uh, subjective. So it's like tailored to an individual person, whereas that's very different in my view from them saying this is something that like we don't approve of as the controllers of this conversation. You see the difference there?

It's very different. Yeah. I mean, if you look at YouTube, Google, Facebook, and Twitter, you have 4 individuals controlling what the world sees based on one ideology of viewpoint.

Yeah. I don't think that the CEOs of those companies are sitting there directly controlling the algorithm, but it's like, it's a team of people. And your point is the same. They're all pretty much from the same background. They're from the same cities. There might be a little bit of difference of opinion on those teams, but when you consider the fact that these platforms have millions, tens of millions, hundreds of millions of people on it, that's nothing. They're basically all the same.

Would we end up in a world where we're all just reading different stuff and we all live in our own individual bubble?

I mean, hasn't it always been like that? I mean, what, what world is it where everyone's reading the same stuff?

In a news world. I mean, that's how it used to be.

Yeah.

Yeah, I guess so.

You didn't have to read the newspaper, but, uh, or is that the problem that's happening? Like consensus of thought. Right, was like we all believed in a certain viewpoint to get aligned around, even capitalism, etc. And then in the internet era, but not everything is an individual interest.

Yeah, but not everyone reads the same newspapers. So now we are going to have differences. It's just the geography is going to matter a lot less. One of the ideas of behind recommendation engines that I really like that could be very applicable here is that you don't want to just be recommended the same old stuff that it kind of knows you like. I kind of think of it as like a kind of concentric circles, right? In the middle, there's the stuff that we know you like. That's kind of your boring filter bubble. If you go way outside of the circle, it's going to be something that's so outside your realm of understanding or what you like, seeing that idea is not not worth your time, but there's also like the periphery of your circle. There's sort of like the edge where it's like, this is something that is close to the things that you've liked before, but it's not quite that. It's a little bit outside your comfort zone. And I think that's whether you're recommending like a restaurant or you're recommending a news article or an opinion, that's the area that's interesting to people. That's the area where you, you learn the most. That's a place to focus on. I know one of my professors at NYU, Alex Tuzhilin, has done a little bit of work. He works on, you know, kind of the theoretical side of recommender systems a lot, and he has a few papers on that out about finding that kind of unexpectedness metric. I think that's sort of what will get people out of their bubble, and I think people enjoy that, actually. I don't think people like staying in their bubble that much.

Can you say more about that? The unexpectedness metric? That sounds fascinating.

Yeah. Let's say like I go, I wake up every morning, I go to Dunkin' Donuts, right? If I wake up and MarsBot says, hey, I have a good idea for you today. Why don't you go to Dunkin' Donuts? Well, I might take that suggestion, right? But I'm not going to be too impressed with MarsBot. But if it sort of shows me an interesting shop around the corner that maybe it doesn't know if I'll like it. You know, give it a try, this is kind of new for you, it's a little bit of an adventure, then yeah, it's a more exciting recommendation. If I'm reading the same stuff every day and I have a recommendation to read the same sort of stuff that I just read the last 5 days in a row, I don't think people are going to enjoy that as much. Even if the new recommendation creates a little conflict, it'll be just outside your realm of debate that it's interesting to you and you can kind of understand where they're coming from.

Is what's happening here that because the platforms became so mature that we've lost the unexpected recommendation that they were when they started?

Yeah, it's amazing. Like in the early days of the Foursquare recommendation engine, how people would attribute, you know, brilliant AI to something that was just, they randomly got something good.

Maybe that's something not to forget about as we move into this AI era. That's a key insight that. If we overdo the AI, we might forget to bring the magic of discovering things we never would have discovered.

Yeah, I mean, that's sort of— I can connect that back to kind of the problem with the idea that there's a singular answer and we just have to throw a lot of math at it and we'll find that optimal answer. That thinking maybe has created a world that we don't like as much, less interesting world.

Maybe you should just remove all of the AI from Foursquare. And send a recommendation daily to everyone and let it be completely random.

I will let some people know that you suggested that.

The new product strategy.

Yeah. Shane Mac mode.

But I think you just said it right. If you put me in control of the algorithm, can I turn on random mode? That's an interesting thing that I hope we all get to explore more. This, this thing you tapped on at the end is how do you bring unexpectedness back to make magic and meaningful moments that are outside of what the algorithm believes is what you want. Yeah. And that's a, that's an interesting area to explore that I haven't thought about really.

Yeah, absolutely.

Machine learning.

Serendipity as a product strategy. We used to call it just living our lives. But now our lives are so influenced by AI that we need to consider the value of an unexpectedness metric and how we can embrace the random. If you're a product designer or marketer looking to improve time on site or app and user satisfaction, you can see how overpolishing the experience might actually wind up giving people a less valuable experience. Heading into our final chapter with Max, we dig into the intersection of privacy with location technology and the question of whether we're facing a potential AI winter. Max Sklar on machine learning from Assist.

With your work at Foursquare, the amount of real-time location tracking has so many privacy implications.

Yeah.

Give me your personal privacy manifesto, like your baseline. What is it?

Well, in my personal internet usage, I don't focus on my personal privacy too much. Maybe I should a little bit more, but I'm on all of the social media. You know, I'm pretty open about which apps I give location to. So our location technology is kind of licensed out to other consumer apps. And I like the idea that each of those apps has a specific reason for having that location technology. For example, we're working with a jukebox app, so it knows when you're in a specific location with the jukebox, then you can kind of play with it. Or a coupon app that reminds you to use the coupon at the right time. And so I like that because there's a reason. I think the negative example would be if some app wasn't using your location for any other purpose other than gathering information on you for, uh, for ads or malware, that kind of thing. You know, I feel like there needs to have some benefit on the consumer side. Also, I appreciate when companies take their security very seriously because there's a lot of— and I'm not an expert in security, but there's always people trying to hack or, you know, trying to get access, particularly when you have a very valuable database.

And you said social. So social's getting hammered right now. There's talk of an AI winner, research and investment waxes and wanes. AI, machine learning, conversational commerce. This is really hot right now. Do you think an AI winner is coming?

You know, it's interesting. I spoke about that a few episodes ago on The Local Maximum and I named the episode Enjoying the AI Summer because I feel like AI is very big right now. It's like a hot, topic, right? Valuations are high. I guess kind of the idea of AI as a chatbot maybe comes and goes a little bit, but machine learning engineers, very high demand. And so there hasn't really been the fall and the winter yet. But yeah, I mean, this thing always comes in cycles. Ultimately, we are in the information age. Information is very valuable. Information processing is very valuable. So Even if there's a down cycle, I'm pretty confident it'll come back.

In the area of AI, where do you think would be the first to experience an AI winner? Where, where is the hype right now?

Hmm. I think that there was a lot of hype in terms of consumer applications. So if, if you talk about consumer chatbots or consumer applications for image recognition, like, let, let me give you an example. I was thinking the other day, if Google Photos— they have a system where they can automatically caption a photo. Like, if you take a photo of a dog, you know, chasing after a ball, then it will caption your photo, "A dog chasing after a ball." Now, if Google Photos said, "Hey, we just upgraded Google Photos, and now we've captioned, automatically captioned all your photos," I would be like, "Awesome, that's really cool." But if they wanted me to pay a significant amount, or if another service came along and said, hey, just pay like, you know, $20 a month and your photos will be captioned, I don't think a lot of people would be that interested in it. They might be like, oh, let me just caption it myself. Or, you know, maybe if the price point is really low, but that's not— it might not be terribly profitable. So I think that the hype might be on the consumer side because that's sort of what's interesting to people from like a media perspective. You immediately understand applications if they're relevant to you personally, like, oh, my photos look like captioned. The things that don't get as much hype that are not relevant unless you're in a specific industry is like industrial applications or agricultural applications or just sort of general business-to-business type applications, which are sort of don't always make as good of a news article, but are probably where the biggest opportunity lies.

Can you give me an example of a really great use of AI that no one's ever heard of?

Hmm. You know, I think that's something that we've talked about, but is sort of very buried at Foursquare, buried in terms of like what our recommendation system does in the popular imagination, is how we built up this system of tastes where we have all this natural language processing that identifies all of the noun phrases that people are talking about in Foursquare. So that's like a lot of the menu items, a lot of the things that you do at places, and that gives us the kind of context, the kind of understanding to be able to tell people what exactly to do at a specific place. And so that use of ML was really interesting from the recommendation standpoint. In terms of our core tech, A lot of machine learning has gone into understanding where people are. And you might not necessarily realize it. You might think, oh, it's just venue shapes, right? Like, you get someone's latitude and longitude, and that says that they're at this Starbucks coffee shop. Like, why would that be so hard? But it turns out that the phone signals are actually really complicated. They jitter around a lot depending upon where you are, what time of day it is. —there's all sorts of signals that go into that designation to try to figure out exactly, you know, whether you're in this shop or whether you're in the shop next door. And I don't think people realize that we can do that, but they also realize that it's a hard kind of machine learning problem.

The use case for Foursquare is so insane, like so many people, locations, pronunciation variations, correcting the errors is a— huge part of the job. What have been your biggest surprises with programming for natural language applications?

NLP is like— I think one of the things he said is that kind of summed it up when I did an episode in NLP, and it was like, with NLP, there's always something. I don't know how to speak any languages other than English, but I got to learn a lot about lots of different languages, and it turns out that anything you think you know about languages, there's always some exception. One example that I had was like with Russian, like it turned out that you can't just get singular or plural. You have to know exactly how many you're talking about or what the ones digit is in some cases. And the frustrating— Is that true?

Yeah. The Russian language, every single quantity amount changes the pronunciation? Well, yeah.

I mean, it's based on like the ones digit, you know, there's, it's not like there's an infinite number of pronunciations, but there's some languages that are just one and many like English. There's There's some that are one, two, and many, and then there are some where, you know, you have to gender it, and there's some where you have to know what the ones digit is. There's all sorts of crazy stuff out there. And so with internationalization and NLP, language is such a hard nut to crack because there's so many exceptions to every rule, and it's amazing how the human brain just kind of gets it.

What needs to be cracked then for, like, the next big leaps forward? Where's the breakthrough going to happen? If the complexity is so high?

I don't know if the goal is necessarily to build a machine that can hold a conversation with a human as a human just yet. I think that from what I've read, the natural language understanding of a document, like summarizing a document and getting information from it, it's still very early in terms of being able to do that as well as a human can. Like, yes, a machine can, you know, scan the entire internet or all of Wikipedia and get all sorts of information, but being able to add to its database with like every sentence, what exactly that sentence means, there's a whole lot of improvement on that, which I'm sure the big tech companies like Google are working on. Thank you for your time today. Yeah, thanks a lot. It's really great to finally be on Machine Learning and join your podcast. And these discussions are always really fascinating because I feel like I'm not telling the audience This is what the answer is. It's like we kind of figure out some ideas and it's like, oh, now I got to go back and kind of maybe try to put some of these into action and see how it looks. Totally. M-A-C-H-I-A-E-L-E-R-X-I-A-E.

Max Sklar and Shane Mac from Assist in another wide-ranging check-in on the technology driving cultural change. AI, social, privacy, NLP, and machine learning. It was great featuring Max this week, and make sure you check out his pod, The Local Maximum. This is the long-form version of the pod. Do you know we release segments of every episode to make it easier to share key ideas with fellow AI and voice professionals? It's true. You can find them where you got this one. Google, iTunes, Spotify, everywhere. Let us know. If you dig this, if it's valuable to you, say so. DMs are always open. You can find us on Twitter @machineypodcast. Please subscribe so you never miss an episode and share this with someone who cares about how we make sense of these changing times. Machine Learning by Assist is made by Paul Ciuffo and Michael Alsesser for Limina House. Have a great day.