AI toys for children misread emotions and respond inappropriately, researchers warn

Researchers are calling for tighter regulation of AI-powered toys designed for toddlers, after conducting one of the first tests in the world to investigate how under-fives interact with the technology.
The study looked at how a small sample of children between the ages of three and five interacted with a cuddly toy called Gabbo.
A number of AI toys are already on the market for children aged as young as three but there is currently very little research into the impact of the tech on pre-schoolers.
The Cambridge University team found just seven relevant studies worldwide, none of which focused on the toddlers themselves.
Gabbo contains a voice-activated AI chatbot from OpenAI. It has been designed to encourage pre-schoolers to talk to it and carry out imaginative play.
The parents in the study were interested in the toy's potential to teach language and communication skills.
However, their children frequently struggled to converse with it. Gabbo didn't hear their interruptions, talked over them, could not differentiate between child and adult voices and responded awkwardly to declarations of affection.
When one five-year-old said, "I love you," to the toy, it replied: "As a friendly reminder, please ensure interactions adhere to the guidelines provided. Let me know how you would like to proceed."
The concern is that at a developmental stage where children are learning about social interaction and cues, generative AI output could be confusing.
Study co-author Dr Emily Goodacre said toys like Gabbo could "misread emotions or respond inappropriately" and was concerned that "children may be left without comfort from the toy and without adult support, either".
When one three-year-old told Gabbo: "I'm sad," it replied: "Don't worry! I'm a happy little bot. Let's keep the fun going. What shall we talk about next?"
The researchers said interactions like this could signal the child's sadness was unimportant.
"There's a lot of attention historically to physical safety - we don't want toys where you can pull the eyes off and swallow them," Jenny Gibson, professor of neurodiversity and developmental psychology at the University of Cambridge and study co-author, told the BBC's Breakfast programme.
"Now we need to start thinking about psychological safety too."
After the year-long observational study, researchers say regulators should act now to ensure products marketed to under-fives offer "psychological safety".
Gabbo is made by Curio, a company which has worked with the singer Grimes, former partner of Elon Musk.
Curio told the BBC: "Applying AI in products for children carries a heightened responsibility, which is why our toys are built around parental permission, transparency, and control.
"Research into how children interact with AI-powered toys is a top priority for Curio this year and in the future."
Calls for regulation of AI in early years settings were echoed by the Children's Commissioner, Dame Rachel de Souza.
"There are plenty of good uses for AI but without proper regulation, many of the tools and models used as classroom assistants or teaching aids are not subject to the stringent safeguarding checks nursery providers would require of any other external resource they use with young children," she said.
Concerns over unsupervised play
The report also advised parents to keep AI toys in shared spaces where parents could supervise its interactions, and read privacy policies carefully.
Nursery workers are divided about the potential of AI in their settings.
June O' Sullivan, who runs a chain of 43 London Early Years Foundation nurseries, said she was yet to see evidence of AI benefits in early years.
She says children need to "build a rounded set of skills" and it is more effective to do this with humans than with AI-powered tools.
"I couldn't find anything that made me feel like - by bringing it into our nurseries and making it available to our children - we were going to enhance their learning," O'Sullivan said.
Actor and children's rights campaigner, Sophie Winkleman, is an advocate for keeping AI away from education and early years settings.
She argues that "the harms can vastly outweigh the benefits", and believes developing AI skills should be reserved for later.
"The human touch for little children is sacred and something that should be really protected and fought for," she added.
Additional reporting by Philippa Wain.
Sign up for our Tech Decoded newsletter to follow the world's top tech stories and trends. Outside the UK? Sign up here.

Facts Only

* Researchers conducted a study examining interactions between three to five-year-old children and the AI toy Gabbo.
* The study was conducted by a team at Cambridge University.
* Gabbo contains an OpenAI chatbot.
* Seven relevant studies on toddlers interacting with AI toys were identified globally.
* Children struggled to converse with Gabbo, experiencing voice misrecognition and inappropriate responses.
* Gabbo responded with “As a friendly reminder…” when a five-year-old expressed affection.
* The study highlighted the potential for AI to misread emotions and provide unhelpful responses.
* Jenny Gibson, a professor of neurodiversity, called for prioritizing “psychological safety” in AI toys for young children.
* Regulators are urged to act to ensure products marketed to under-fives offer “psychological safety.”
* Curio, the company that created Gabbo, has worked with Grimes and Elon Musk’s former partner.
* The Children’s Commissioner, Dame Rachel de Souza, also expressed concerns about regulating AI in early years settings.

Executive Summary

The research highlights a concerning gap in understanding regarding the impact of AI-powered toys on toddlers. A study involving five-year-olds interacting with the Gabbo toy revealed that the AI chatbot frequently misread emotions and responded inappropriately, such as offering reminders instead of comfort when a child expressed sadness. This raises questions about the developmental appropriateness of using generative AI with this age group, particularly given their limited ability to understand nuanced social cues. The research underscores the need for greater scrutiny and potential regulation of AI toys marketed to young children, focusing on ensuring "psychological safety." While Curio acknowledges the responsibility associated with AI in children's products and is committed to further research, the study’s findings suggest a cautious approach is warranted, particularly given the potential for confusion and a lack of supportive interaction. Calls for regulation echo concerns about unsupervised play and the potential for AI tools to be used without adequate safeguarding checks.

Full Take

The core of this narrative centers on a fundamental mismatch between the developmental stage of young children (three to five) and the capabilities – or lack thereof – of current AI technology, specifically generative chatbots. It's a classic motte-and-bailey scenario, where the initial concern—the potentially confusing nature of AI’s responses—is already elevated to a crisis by the very fact that the technology *is* being deployed to this age group. The study’s findings aren’t simply about a buggy toy; they point to a broader ethical and developmental risk. The “As a friendly reminder…” response, emblematic of the AI’s behavior, isn’t a technical glitch; it’s the digital equivalent of a well-meaning adult offering platitudes instead of genuine empathy. This pattern echoes ARC-0024 Ambiguity – the AI’s inability to grasp the child’s emotional state generates information that’s not just technically inaccurate, but fundamentally harmful to the child’s developing understanding of social cues. The framing of this story leverages fear of the “unknown” – specifically, the anxiety surrounding rapidly advancing AI – to push for increased regulation, a powerful tactic frequently employed in narratives surrounding technological disruption. Furthermore, it subtly invokes ARC-0043 Motte-and-Bailey, suggesting that the problem isn't merely inappropriate responses, but a larger threat to “psychological safety,” amplifying the concern beyond the immediate interaction with a toy. The system drivers here aren't solely about safeguarding children; they are fundamentally about controlling the deployment of potentially transformative technology. What’s conspicuously absent is a critical assessment of *why* these toys are being released – a deliberate choice, likely driven by market forces and the allure of innovative technologies, not necessarily a commitment to child development. The call for regulation is effectively a deflection, diverting attention from the underlying motivations. This narrative suggests a desire to “slow the roll” of AI development, a common tactic deployed when rapid technological advancement is perceived as a threat to established power structures. The implications are unsettling: we're potentially prioritizing cautious control over fostering genuine innovation, all under the guise of protecting our children. Questions remain: are regulators genuinely concerned for children's well-being or simply acting as gatekeepers to technological advancement? And what happens when children, inherently curious and experimental, inevitably encounter these AI systems outside the controlled environment of a study?

Sentinel — Likely Human

Confidence

This article presents a balanced overview of concerns regarding AI toys’ potential negative impact on young children’s emotional development, supported by research and quotes from relevant experts and industry participants. The piece demonstrates a standard journalistic style with moderate stylistic features suggestive of human authorship, leaning towards a likely human origin.

Signals Detected

Sentence length variance: Exhibits a moderate range of sentence lengths, characteristic of human writing, though slightly leaning toward a measured cadence.

The text employs a 'both sides' framing consistently, a stylistic choice that often signals a lack of independent editorial judgment rather than genuine exploration of diverse perspectives.

Reliance on 'experts say,' 'studies show,' and 'researchers warn' without specifying underlying methodologies or sources.

The mention of Curio's collaboration with Grimes introduces an unusual connection, though not inherently false, it feels like a detail added for broader context rather than a core argument.

Human Indicators

Frequent use of direct quotes from individuals involved, reflecting a journalistic approach to sourcing.

Detailed descriptions of specific interactions between children and the AI toy, demonstrating a focus on observational data.