A 13-year-old girl spent three months using her school-issued laptop to discuss depression, research suicide, and plan her funeral before self-harm-monitoring software finally prompted school officials to alert her parents, who intervened and stopped her, according to a lawsuit filed this month by her parents.
The lawsuit alleges that Google and parent company Alphabet were negligent in their rollout of a Gemini model, claiming that the AI chatbot “coached” the North Carolina teenager to consider taking her own life. The case, which was previously reported by The New York Times, is one of several that allege harm from AI chatbot use, including claims of delusions and suicide. But it also illustrates the limitations of surveillance tools that thousands of schools have adopted in recent years, which are ostensibly designed to protect students from themselves when browsing and chatting online.
Andrew Liddell, a lawyer for the family, tells WIRED that, according to information the school provided to the family, the alert went to the personal phone of a former counselor who had left the school nine months before. The ex-employee was diligent in passing the alert along to current staff, who then notified the girl’s mother, he says.
As classrooms continue to rely on Google’s low-cost offerings, including Gemini, their ability to keep up with the potential risks is front and center. Just last week, parent and teacher groups across the country decried Google’s new requirement that schools must turn on generative AI features if they want to use Search. US senator Ed Markey cited Emma’s lawsuit in a letter to Google demanding the company let schools disable AI Mode and AI Overviews, writing that “Google is recklessly forcing AI on schools and treating kids like collateral damage.”
According to the lawsuit, the North Carolina girl, identified as Emma Doe, descended into “a severe psychological crisis” after Google Search routed her to Gemini on her school-issued Google Chromebook. Though Emma’s school had blocked the stand-alone Gemini website, the lawsuit alleges that she was steered into interacting with the chatbot through AI Mode, which lets people ask follow-up questions about a query or begin an unrelated conversation.
Emma queried AI Mode about mental health issues starting last December. She asked about “severe depression” and “reasons to live,” and by late January she was asking about suicide, per the lawsuit. Over the course of two days around Valentine’s Day, she began writing a 1,500-word eulogy for herself with help from AI Mode in a Google Doc titled “Funeral.” The lawsuit says she worked on the document for more than four weeks while continuing to interact with the Gemini tool thousands of times.
After school on St. Patrick's Day, she deleted most of the file and restarted. But Liddell says the themes and thoughts the girl expressed didn't materially change. Around 7:30 pm, her mother allegedly received a message from the school indicating that her daughter was in “imminent danger,” the lawsuit states. Her mother used her phone to log in to Emma’s school account and “watched her daughter compose a suicide note in real time,” according to the lawsuit.
Google spokesperson José Castañeda says that the company is seriously “reviewing the claims in the lawsuit” and that its systems, including AI Mode, “aim to prevent users from engaging in harmful conversation.”
"We’ve worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it,” Castañeda says.
Lawyers for the family have sought to withhold the school's name to protect Emma's privacy. WIRED was able to identify the school, which did not return requests for comment. In recent months, the school added links to suicide prevention resources to the bottom of its website, according to WIRED’s review of the Internet Archive.
Liddell tells WIRED that Emma’s school only learned about her mental health crisis because it used at least one third-party student-activity-monitoring system. Citing information that was allegedly previously provided to the school’s parents, he says these may have included products from the companies GoGuardian and Gaggle. Liddell says that the family lacks a definitive answer because the school’s “window of cooperation” with them was brief.
GoGuardian spokesperson Natalia Switala tells WIRED that the company cannot “discuss any individual student or district” due to “student privacy obligations.” Gaggle did not immediately return WIRED’s request for comment.
The screening tools generally work by capturing keystrokes or scanning files and using algorithms and human review to identify potentially concerning content. It’s unclear whether the school’s third-party screening tools monitor AI Mode interactions or, if they do, how well they actually work. GoGuardian wrote in a July 2025 blog post that its flagship product monitors conversations with AI chatbots, including Gemini, but it didn’t mention AI Mode specifically.
Though it’s unclear exactly how effective student surveillance tools are at protecting students, studies show that educators perceive them as helpful. A 2022 white paper from the EdWeek Research Center found that in a survey of 1,000 American K-12 educators, 96 percent of Gaggle users believed that it helped to “keep students safe from suicide and/or self-harm.” While the study was sponsored by Gaggle, the findings were similar for teachers who reported using alternatives.
The allegation that Emma’s escalating interactions with Gemini went unnoticed for almost two months, after she explicitly asked about “hanging, suffocation, and overdosing,” appears to indicate that chatbot conversations embedded in Google Search may be a blind spot.
Civil rights groups have been critical of student surveillance tools, including calling out the marketing tactics and the invasion of student privacy. Experts have also worried that surveillance software undermines students’ ability to trust their teachers, the very people who may be able to help a child in crisis.
Benjamin Boudreaux, a policy researcher at Rand, which studies topics like student surveillance, tells WIRED that he fears that some schools have rushed to use tools without considering their limits and proper processes to act on alerts.
He expects Emma’s lawsuit to spark further outcry from parents, teachers, and students for instructional opportunities free of AI and surveillance and full of empathy. “It’s really only through that human connection,” he says, “that we can address the root issues.”
