Robert TaFounder & CEO, Clarity · Building Hyper-Personalized Agents

AI Learned Scientific Taste & Beat GPT-5.2: RLCF vs RLHF Explained

6 Apr 2026 · 2 min

Watch on YouTube

1:48

In this commentary, the speaker outlines a recent research paper titled "AI Can Learn Scientific Taste," which demonstrates that an AI system can be trained to judge the intrinsic value of scientific ideas rather than merely ranking or preferring them. The researchers built a judge model using 700,000 matched pairs of high‑citation and low‑citation papers from the same fields and time periods, enabling the model to predict which work will have greater impact. This judge model then served as a reward signal for a generator (the "thinker") trained via Reinforcement Learning from Community Feedback (RLCF), a paradigm distinct from traditional Reinforcement Learning from Human Feedback (RLHF) that relies on individual annotators. The resulting system reportedly outperforms GPT‑5.2 in generating higher‑impact research ideas. The speaker emphasizes that "taste" reflects a compression of community beliefs into a reward signal, allowing AI to produce content that aligns with what the scientific community values, not just more content. The discussion positions this work as a shift from quantity‑focused generation toward quality‑focused generation grounded in collective scientific judgment.

Chapters

Researchers just trained an AI to have taste. Not preferences, not rankings, actual taste. The ability to judge whether an idea is actually good, and it outperformed GPT-5.2 at it. The research paper is called "AI Can Learn Scientific Taste." They trained a judge model on 700,000 matched pairs of high-citation versus low-citation papers. Same field, same time, but a very different impact. The model learned to distinguish papers that would matter from papers that wouldn't. Then they used that judge as a reward signal to train a thinker that generates research ideas with higher predicted impact. They called it Reinforcement Learning from Community Feedback, RLCF, not RLHF with individual annotators. What the collective scientific community values encoded as training signal. The judge models what a community believes is valuable. Everyone's building AI that generates more, more, more, more, more text, more code, more content, more images, more videos. And this paper asks a very different question: can AI learn what's worth generating in the first place? My take is that taste is based on beliefs. It's a compression of what matters into a reward signal. The AI that has taste doesn't just produce more, it produces what matters aligned to the goal, and in this case, the users or the scientific community. Taste, not preferences, not rankings, the ability to actually judge what's good. And they proved AI can learn it.