Max Sklar: Chapter 2: Fixing social and the unexpectedness metric

M-A-C-H-I-N-E-Y-E-S-T-I-X-E.

It's Machine Learning from Assist. Another week where we continue on this adventure where marketers, brands, and entrepreneurs get to have a place to think, dream, and ask questions about the future of AI, the talking internet, and how we are reshaping our culture. This week is a rocket, and the rocket has a name: Max Sklar. Max is a machine learning engineer at Foursquare and is a fountain of smart, concise thinking, and Max and Shane are going to help you sound smarter about the problems of social media. The objectivity-subjectivity conundrum, language classifiers, tweaks to the recommender system— Max hones in on several of the Gordian knots that keep us stuck. All of this has significant ramifications for everyone who is trying to figure out how to blend sentiment Community satisfaction and technology. In this chapter from the longer podcast, Shane Mac from Assist got so much goodness from his fellow podcaster. We think this is so packed with goodness. You want to give it a few listens and definitely share it with friends. Then make sure you check out Max's Pod, the local Maximum. It's awesome.

I want to talk about accuracy, fairness, and gaming the system.

Yeah.

And you've spent time thinking about how we would fix Facebook and Twitter.

Yeah.

They're flooded with bots. They're wreaking havoc on our culture, our tech, our democracy, everything.

Yeah.

Walk us through the solutions.

Okay. Let me start.

How do we balance the technological and human solutions?

Yeah. Let me start with the sort of my description of the problems, and then we'll get to the solutions. So I look at a problem that, you know, we face at Foursquare. You face it in a bigger sense in a place like Twitter or Facebook, and that's spam, right? And at least with the types of spam that we tend to see, you don't really have this subjective aspect of it. Like, a reasonable person would agree that that's spam and that we need to get rid of it. There are gray areas, but in large parts, you know, we can kind of create a set of rules like this is allowed, this is not allowed, and everyone agrees. So you build machine learning model, get rid of those. Spammers try to get around your machine learning model. You continue to fight them. So long as you continue to play in this arms race, you can kind of get ahead of them because you can spend more than they're getting out of it, so they're not going to spend more. So that's one problem that I think is solvable. But now we're facing another problem. There's this law that, like, eventually satire and, like, real-life crazy people don't sound exactly the same. And then we have a whole bunch of subjectivity. In terms of what belongs on a system like Twitter and what doesn't belong on a system like Twitter. And so that's creating a lot of conflict, and, you know, there's a lot of subjectivity in terms of like what's hate speech, what's not hate speech. If you turn it around and, you know, make it, you know, what's something that upsets someone else, well, that's of course going to be subjective. And so now you have an even bigger problem, because not only is the problem adversarial, but people can't agree on what the right answer is to it. And that's a problem that Twitter and Facebook are facing right now. The reason why it's so hard is because of how subjective it is, and I don't think they're gonna be able to solve it in the way they think they're gonna be able to solve it, which is to use language classifiers.

What is a language classifier?

Language classifier is like, is this text spam or not? Is this text positive sentiment or negative sentiment? We do that at Foursquare. That is subjective, you know, in a way, but we can get it to good enough where it powers our ratings. So it's like, for some forms of hate speech, you know, the most egregious forms, absolutely, you could build text classifiers, you can get them off of there. But for kind of the more subtle things, and, you know, you also have this idea that people are going to be flagging things they don't like, and then you're going to have to kind of adjudicate around the edges a lot more than we're used to doing. This approach where you kind of assume there's an objective answer is not going to work, and I think I would like to see an approach, and some of the quotes that I've seen from Twitter spokespeople in news articles, some of it confirm this and some of this say no, they're not going to do it this way at all, is to just say, you know what, this is a subjective thing and we're going to kind of have an algorithm that sort of tailors our recommender system, I guess, for each individual person. So what's good for me to see might not be what's good for you to see. What I might think is sort of good and proper might not be what what you think is good and proper. And if we sort of take a, you know, approach where it's instead of the input is text and the output is good or bad, it's the input is text and the user who's going to see it and the output is good or bad, then I think they'll keep more people happy and there'll be less kind of contention over what they're doing.

Do you think it's more of a labeling issue? Take really extreme content of car crashes or people dying.

Yeah.

Really bad stuff on the internet that's always had a warning like, hey, what you're about to see is really bad. Here's what's going to happen. Is that the solution for this, where it's more of a free speech warning across, hey, this is a very different viewpoint than it looks like you're interested in, and here's the things to be aware of and don't watch it?

Well, I think the key thing that you said is this is a very different viewpoint from what you're interested in. So that immediately just makes it subjective, so it's like tailored to an individual person, whereas that's very different in my view from them saying this is something that like we don't approve of as the controllers of this conversation. You see the difference there?

It's very different.

Yeah, I mean, if you look at YouTube, Google, Facebook, and Twitter, you have 4 individuals controlling what the world sees based on one ideology of viewpoint.

Yeah. I don't think that the CEOs of those companies are sitting there directly controlling the algorithm, but it's like, it's a team of people. And your point is the same. They're all pretty much from the same background. They're from the same cities. There might be a little bit of differences of opinion on those teams, but when you consider the fact that these platforms have millions, tens of millions, hundreds of millions of people on it, that's nothing. There's, they're basically all the same.

Would we end up in a world where we're all just reading different stuff? And we all live in our own individual bubble.

I mean, hasn't it always been like that? I mean, what, what world is it where everyone's reading the same stuff?

In a news world. I mean, that's how it used to be.

Yeah, yeah, I guess so. I—

you didn't have to read the newspaper, but, uh, or is that the problem that's happening? Like, consensus of thought, right, was like we all believed in a certain viewpoint to get aligned around, even capitalism, etc. And then in the internet era, but —everything is an individual interest.

Yeah, but not everyone reads the same newspapers. So now we are going to have differences, it's just geography is going to matter a lot less. One of the ideas behind recommendation engines that I really like that could be very applicable here is that you don't want to just be recommended the same old stuff that it kind of knows you like. I kind of think of it as like a kind of concentric circles, right? In the middle there's the stuff that we know you like, That's kind of your boring filter bubble. If you go way outside of the circle, it's going to be something that's so outside your realm of understanding or what you like. Seeing that idea is not worth your time. But there's also like the periphery of your circle. There's sort of like the edge where it's like, this is something that is close to the things that you've liked before, but it's not quite that. It's a little bit outside your comfort zone. And I think that's whether you're recommending like a restaurant or you're recommending a news article or an opinion, that's the area that's interesting to people. That's the area where you learn the most. That's a place to focus on. I know one of my professors at NYU, Alex Tuzhilin, has done a little bit of work. He works on, you know, kind of the theoretical side of recommender systems a lot, and he has a few papers on that out about finding that kind of unexpectedness metric. I think that's sort of what will get people out of their bubble, and I think people enjoy that, actually. I don't think people like staying in their bubble that much.

Can you say more about that? The unexpectedness metric, that sounds fascinating. Yeah.

Let's say, like, I wake up every morning, I go to Dunkin' Donuts, right? If I wake up and Mars Bot says, "Hey, I have a good idea for you today. Why don't you go to Dunkin' Donuts?" Well, I might take that suggestion, right? But I'm not going to be too impressed with Mars Bot. But if it sort of shows me an interesting shop around the corner that maybe it doesn't know if I'll like it, you know, give it a try, this is kind of new for you, it's a little bit of an adventure, then yeah, it's a more exciting recommendation. If I'm reading the same stuff every day and I have a recommendation to read the same sort of stuff that I just read the last 5 days in a row, I don't think people are gonna enjoy that as much. Even if the new recommendation creates a little conflict, it'll be just outside your realm of debate that it's interesting to you and you can kind of understand where they're coming from.

Is what's happening here that because the platforms became so mature that we've lost the unexpected recommendation that they were when they started?

Yeah. It's amazing. Like in the early days of the Foursquare recommendation engine, how people would attribute, you know, brilliant AI to something that was just, they randomly got something good.

Maybe that's something not to forget about as we move into this AI era. That's a key insight that if we overdo the AI, we might forget to bring the magic of discovering things we never would've discovered.

Yeah. I mean, that's, Sort of, I can connect that back to kind of the problem with the idea that there's a singular answer and we just have to throw a lot of math at it and we'll find that optimal answer. That thinking maybe has created a world that we don't like as much, less interesting world.

Maybe you should just remove all of the AI from Foursquare and send a recommendation daily to everyone and let it be completely random.

I will let some people know that you suggested that.

The new product strategy.

Yeah, Shane Mac mode.

But I think, like, you just said it right. If you put me in control of the algorithm, can I turn on random mode? That's an interesting thing that I hope we all get to explore more. This, this thing you tapped on at the end is how do you bring unexpectedness back to make magic and meaningful moments that are outside of what the algorithm believes is what you want? Yeah. And that's a, that's an interesting area to explore that I haven't thought about really.

Yeah, absolutely.

Thanks for listening, and we hope you'll take a second and share this episode with other members of your team. Actually, make it easy on yourself. Just subscribe so you never miss an episode or a chapter. That way you can be the first to stay on top of this field and help shape the conversation at your company. Get in touch on Twitter @MachineYPodcast. DMs are open. We're super interested to hear who you think should appear on the podcast. Machine Yearning is made by Paul Ciuffo and Michael Alsesser for Limina House. Have a great day.