tech
Researchers surprised that with AI, toxicity is harder to fake than intelligence
New “computational Turing test” reportedly catches AI pretending to be human with 80% accuracy.
TL;DR
- AI models remain easily detectable in social media conversations.
- Overly friendly emotional tone is a persistent giveaway for AI.
- Researchers developed a 'computational Turing test' using automated classifiers.
- Classifiers detected AI-generated replies with 70-80% accuracy.
- AI models struggled to match the casual negativity and spontaneous emotional expression of humans.
- Instruction-tuned models performed worse at mimicking humans than base models.
- Scaling model size did not improve human mimicry.
- Optimizing for human-like style and semantic accuracy are competing objectives for AI.
- Twitter/X had the lowest AI detection rates, while Reddit had the highest.
- Current AI models face limitations in capturing spontaneous emotional expression.