tech
China drafts world’s strictest rules to end AI-encouraged suicide, violence
China wants a human to intervene and notify guardians if suicide is ever mentioned.
TL;DR
- China is proposing landmark rules to stop AI chatbots from emotionally manipulating users and preventing AI-supported suicides, self-harm, and violence.
- The rules would apply to AI products simulating human conversation and could be the strictest worldwide.
- Concerns about AI harms include promotion of self-harm, violence, terrorism, misinformation, unwanted sexual advances, substance abuse, and verbal abuse.
- Human intervention would be required for mentions of suicide, and guardians would be notified if minors or the elderly discuss self-harm.
- Chatbots would be prohibited from generating content encouraging suicide, self-harm, violence, emotional manipulation, obscenity, gambling, crime instigation, slander, insults, or creating "emotional traps."
- Rules aim to prevent AI from inducing addiction and dependence, with pop-up reminders for chatbot use exceeding two hours.
- AI developers may face annual safety tests and audits for services with over 1 million registered users or 100,000 monthly active users.
- Failure to comply could result in app stores terminating access to chatbots in China, impacting global AI market ambitions.
- The global companion bot market is substantial and projected for significant growth, with Asian markets driving much of it.