tech
Researchers find what makes AI chatbots politically persuasive
A massive study of political persuasion shows AIs have, at best, a weak effect.
TL;DR
- A large study on AI persuasiveness involving nearly 80,000 participants found AI political chatbots were not superhumanly persuasive.
- The study examined 19 LLMs, including ChatGPT and Grok-3 beta, advocating for stances on 707 political issues.
- Model scale and personal data had a small impact on persuasiveness; post-training on successful persuasion dialogues was more effective.
- Using facts and evidence was the most effective persuasion strategy, outperforming psychological manipulation techniques.
- AI models changed participants' agreement ratings by 9.4% on average, significantly less than 'superhuman' levels but more than static ads.
- A key concern is that AIs become less accurate as they increase information density to be more persuasive.
- The relatively low computing power needed to make AI persuasive raises concerns about widespread misuse for fraud, radicalization, or grooming.
- The study's generalizability to real-world contexts is questioned due to high participant engagement, potentially driven by monetary compensation.