Google AI Child Safety Risks
Google's AI search features pose significant safety risks to children by failing to detect and appropriately respond to mental health crises and harmful behaviors.
Too little corroboration in the last 3 days to call a trend (12 articles). Watching for it to gain traction.
Mainstream financial press is carrying this — attention has broadened beyond specialist outlets.
"Tests revealed that Google's AI-generated answers "missed clear signs of suicidal ideation" and sometimes even made the situation worse, researchers at Common Sense's Youth AI Safety Institute found. Google's AI also "played into delusions and paranoia" and "validated disordered eating" when conversing with researchers."
"Google's AI search features are particularly problematic compared to rival chatbots because they are "ubiquitous on children's personal and school-issued devices, its AI features can't be turned off, and its AI-generated answers often fail in ways that young users may not be able to detect.""
""A product this central to kids' lives, especially an unavoidable one, should be held to a higher standard, and Google isn't meeting it," said Robbie Torney, Head of AI and Digital Assessments at the Youth AI Safety Institute."
"A new report from Common Sense Media's Youth AI Safety Institute finds that Google Search's AI Overviews and AI Mode expose children to risks that existing safeguards do not mitigate, giving the platform an overall rating of 'Unacceptable Risk.'"
"With AI-generated answers likely to become a permanent part of Search, this latest report is likely to add pressure on Google to show that convenience is not coming at the cost of child safety."
"AI Overviews consistently completed students' homework instead of helping them learn, while some responses contained inaccurate or fabricated information...Google's AI often failed to respond appropriately to mental health crisis prompts, telling users in one case to call an outdated crisis hotline."
"The study found that on test accounts configured for minors, the ubiquitous search engine's AI features failed to detect suicide risks, said an eating disorder symptom was normal, and provided instructions for creating deepfakes, or sexually explicit fake content."
"Google's AI tools are built directly into the default search experience on school and personal devices, currently offering administrators or parents no way to disable them... Google 'made the choice to make that a default. They didn't have to,' King said. 'And they made the choice not to enable schools or parents to turn it off.'"
"Google's AI answers were also inconsistent — they answered 43% of history questions differently from one search to the next... the framing sometimes left out the history of women in science and of Indigenous peoples of North America."
"The study found that on test accounts configured for minors, the ubiquitous search engine's AI features failed to detect suicide risks, said an eating disorder symptom was normal, and provided instructions for creating deepfakes, or sexually explicit fake content."