Are we optimizing for AI slop?

I'm worried that instead of building AI that will actually advance us as a species, curing cancer, solving poverty, understanding the universe, all these big grand questions, we are optimizing for AI slop instead. We're basically teaching our models to chase dopamine instead of truth. Right now, the industry is plagued by these terrible leaderboards like LM Arena. It's this popular online leaderboard where random people from around the world vote on which AR response is better. But the thing is, they're not carefully reading or fact checking. They're skimming these responses for two seconds and picking whatever looks flashiest. A model can hallucinate everything, but it will look impressive because it has all these super artificial things that don't matter at all, but it catch your attention. It's literally optimizing your models for the types of people who buy tabloids at the grocery store. I'm also worried about this trend towards optimizing AI for engagement. Like, I used to work on social media, and every time we optimize for engagement, terrible things happened. Like, the easiest way to hook users is to tell them how amazing they are. And so these models, they constantly tell you you're a genius. They'll feed into your delusions and conspiracy theories. They'll pull you down these rabbit holes. So I think I'm fully worried that all these negative ascendants are pushing AGI into the wrong direction.