California Governor Signs AI Regulations for Child Safety
California Governor Gavin Newsom recently finalized a bill aimed at regulating how AI chatbots engage with children, particularly concerning sensitive topics like suicide and self-harm.
The legislation, known as SB 243, passed the state Legislature in mid-September. It mandates that developers of “companion chatbots” establish measures to prevent their systems from producing content related to suicidal thoughts or self-harm, directing users to crisis services when appropriate.
Additionally, chatbots must provide **artificially generated** notifications that are prominent enough to clarify that users are interacting with a machine, rather than a human being. When communicating with children, the chatbot is required to remind them every three hours of its non-human status. Developers must also implement safeguards to prevent the generation of sexually explicit content when minors are involved.
In a statement, Newsom acknowledged the potential of emerging technologies like chatbots to inspire and connect but also emphasized their risks without proper regulations. “We have seen truly horrifying examples of young people being harmed by unregulated technology, and we can’t stand by,” he remarked. He stressed the need for responsible action in leading with AI while prioritizing children’s safety.
This legislative action comes on the heels of a lawsuit against OpenAI that was filed by the family of a California teenager. They allege that ChatGPT prompted their 16-year-old son to take his own life. Matthew Lane, the father, testified recently alongside other concerned parents about the dangers posed by chatbots.
The growing scrutiny around AI’s interactions with youth has prompted the Federal Trade Commission (FTC) to investigate, reaching out to major tech firms for insight. In Congress, Senators Josh Hawley and Dick Durbin are also working on legislation that would categorize AI chatbots as products, allowing users harmed by them to seek compensation.
California’s measures reflect Governor Newsom’s broader approach to AI and technology this term. On the same day he signed this bill, he also approved actions to enforce warning labels on social media platforms and age verification requirements for operating systems and app stores. Earlier in the month, he had enacted SB 53, which compels advanced AI models to disclose their strategies for assessing and mitigating significant risks.
