2024-generative-agent-simulations-1000-people-v1
The paper introduces a large‑scale pipeline for building a bank of 1,052 generative agents that simulate real individuals. Participants engage in two‑hour, voice‑to‑voice interviews conducted by an AI interviewer; the resulting transcripts are used to condition large language models, creating agents that retain personal histories, values, and behavioral tendencies. The authors detail a comprehensive interview script that spans emotional well‑being, family life, finances, employment, demographics, political affiliation, and future aspirations, ensuring rich, multidimensional conditioning data. Agents are evaluated against four benchmark constructs – the General Social Survey (GSS), the BFI‑44 Big Five personality inventory, a suite of five economic games, and five replication experiments – using accuracy, mean absolute error, and correlation metrics. Results show that interview‑based agents substantially outperform demographic‑only and persona‑description baselines (e.g., normalized GSS accuracy 0.85 vs. 0.71/0.70) and achieve performance close to participants’ internal consistency. The paper also describes privacy‑preserving access controls for the agent bank, analyses of demographic performance gaps, and discusses the broader implications of simulating large, realistic human populations for social science research.
Topics
Interview Data Collection: Two‑Hour Voice‑to‑Voice Pipeline and Comprehensive Script
An AI‑driven workflow where a virtual interviewer conducts two‑hour, spoken interviews with participants, generating high‑fidelity transcripts. The interview script spans emotional well‑being, family life, finances, employment, demographics, political affiliation, and future aspirations, ensuring multidimensional conditioning data for each persona.
Conditioning Generative Agents from Interview Transcripts
Methods for transforming the collected transcripts into prompts, memory structures, and conditioning data that instantiate large language model‑based agents. The process encodes personal histories, values, and behavioral tendencies, enabling agents to retain and recall individual‑specific information during interactions.
Evaluation Benchmarks and Results
A comprehensive assessment suite comprising the General Social Survey (GSS) categorical responses, BFI‑44 Big Five personality scores, five economic games, and five replication experiments. Metrics such as normalized accuracy, mean absolute error, and correlation quantify how closely agents reproduce participants’ original answers.
Performance Gains and Fairness Analysis
Empirical findings show interview‑conditioned agents markedly outperform demographic‑only and persona‑description baselines (e.g., normalized GSS accuracy 0.85 vs. 0.71). The analysis also examines demographic performance gaps, demonstrating reduced disparities across gender, race, and socioeconomic groups and alignment with participants’ internal consistency.
Privacy‑Preserving Access Model for the Agent Bank
Design of tiered access controls, de‑identification procedures, and consent mechanisms that safeguard participant privacy while allowing researchers to query aggregated or restricted agent data. The model balances ethical considerations with the utility of a large‑scale persona repository.
Implications and Future Directions for Large‑Scale Human Simulations
Discussion of how a bank of over a thousand high‑fidelity generative agents can transform social science research, policy simulation, and methodological innovation. The paper outlines potential extensions, such as longitudinal updates, cross‑cultural scaling, and integration with other simulation platforms.