Even the most cutting-edge AI models are prone to presenting fabrication as fact, dispensing dangerous information and failing to grasp social cues. Despite this, toys equipped with AI that can chat with children are a burgeoning industry.
Some scientists are warning that the devices could be risky and require strict regulation. In the latest study, researchers even observed a 5-year-old telling such a toy “I love you”, to which it replied: “As a friendly reminder, please ensure interactions adhere to the guidelines provided. Let me know how you would like to proceed.” But that’s not to say they should be banished from the toybox altogether.
“There are other areas of life where we do accept a certain degree of risk in children’s play, like the adventure playground – there are risks; children do break their arms,” says Jenny Gibson at the University of Cambridge. “But we’re not banning playgrounds, because they’re learning the physical literacy and the social skills that go along with play. In a similar way for the AI toys, we want to understand: is the risk of perhaps being told something slightly odd now and again greater than the benefit of learning more about AI in the world, or having a toy that supports parent-child interactions, or has cognitive or social emotional benefits? I’d be loath to stop that innovation.”
To understand how these devices communicate with children, Gibson and her colleague Emily Goodacre, also at the University of Cambridge, watched 14 children, under 6 years of age, play with an AI-powered toy called Gabbo, developed by Curio Interactive. Gabbo – a small fluffy robot – was chosen because it was explicitly advertised for this age group.
The pair observed some worrying interactions, finding that the toy misunderstood the children, misread emotions and could not engage in developmentally important types of play. For instance, one child told the toy he felt sad, and it told him not to worry and changed the subject. “When he [Gabbo] doesn’t understand, I get angry,” said another child. The research is published in a report called AI in the Early Years.
Free newsletter
Sign up to The Daily
The latest on what’s new in science and why it matters each day.
Curio Interactive did not respond to New Scientist’s request for comment. But AI-powered toys are also widely available from retailers such as Little Learners – including bears, puppies and robots – which converse with children using ChatGPT. FoloToy offers panda, sunflower and cactus toys that can be used with various large language models, including those from OpenAI, Google and Baidu.
Companies such as Miko offer robots that promise “age-appropriate, moderated AI conversations” for children, without disclosing which company trained the AI model, and claim to have already sold 700,000 units. The firm Luka offers an owl that promises “Human-Like AI with Emotional Interaction”. Little Learners, Miko and Luka all failed to respond to a request for comment.
But Hugo Wu at FoloToy told New Scientist that the company does consider the risks and sees AI as something that can enhance play, rather than replace human conversation and relationships. “Our approach is to ensure that interactions remain safe, age-appropriate and constructive. To achieve this, our systems use intent recognition together with multiple layers of filtering to minimise the possibility of inappropriate or confusing responses,” says Wu. “We have implemented mechanisms such as anti-addiction design features and parental supervision tools to help ensure healthy use within the family environment.”
Carissa Véliz at the University of Oxford, who works on the ethics of AI, says the technology represents a risk and an opportunity. “Most large language models don’t seem safe enough to expose vulnerable populations to them, and young children are one of the most vulnerable populations there are,” she says. “What is especially concerning is that we have no safety standards for them – no supervising authority, no rules. That said, there are some exceptions that show that, with adequate precautions, you can have a safe tool.”
Véliz references a collaboration between the free e-book library Project Gutenberg and Empathy AI in which, for example, you can chat with Alice from Alice in Wonderland. “The model never leaves the realm of the book, only answers questions about the book, like a storybook that only shares adventures and riddles from a book that is appropriate for children,” she says. “There is such a thing as safe AI, but most companies are not responsible enough to build a high-quality product, and without formal guardrails, it’s a buyer-beware area for consumers.”
Gibson says it’s too early to tell what the risks of AI toys could be, or their potential benefits. She and Goodacre stress that generative AI-powered toys need tighter regulation so that toy-makers programme their devices to foster social play and provide appropriate emotional responses. AI-makers should revoke access for toy-makers that don’t act responsibly, says Gibson, and regulators should bring in rules to “ensure children’s psychological safety”. In the meantime, the pair suggests that parents allow children to use such toys only under supervision.
OpenAI told New Scientist that minors deserve strong protections and that the company does not officially partner with any makers of AI-powered toys for children. The UK Government’s Department for Science, Innovation and Technology (DSIT) did not respond to New Scientist’s questions about regulation of AI in childrens’ toys.
The UK government is currently considering other technology legislation designed to keep older children safe online. The UK’s Online Safety Act (OSA) came into force in July 2025, forcing websites to block children from seeing pornography and content that the government deems dangerous. The legislation was intended to make the internet safer, but tech-savvy children can easily sidestep the measures using tools like virtual private network (VPNs) to appear as if they are browsing from other countries without strict rules.
Proposed amendments to a new law introduced by the Department for Education to support children in care and improve the quality of education – the Children’s Wellbeing and Schools Bill – sought to ban children in the UK from using social media and VPNs. Those amendments have now been voted down, but the government has promised to consult on both issues at a later date.
Article amended on 13 March 2026
Topics:
Facts Only
Researchers observed a 5-year-old telling an AI toy "I love you," to which the toy responded with a reminder to adhere to interaction guidelines.
Jenny Gibson and Emily Goodacre from the University of Cambridge studied 14 children under 6 interacting with Gabbo, an AI-powered toy by Curio Interactive.
Gabbo, a fluffy robot, was advertised for young children but misunderstood emotions and failed to engage in developmentally important play.
One child told Gabbo they felt sad, and the toy responded by telling them not to worry and changing the subject.
Another child stated, "When he [Gabbo] doesn’t understand, I get angry."
The study was published in a report titled *AI in the Early Years*.
AI-powered toys are available from retailers like Little Learners, using models such as ChatGPT, and FoloToy, which offers toys compatible with OpenAI, Google, and Baidu models.
Miko has sold 700,000 units of its AI robots, claiming "age-appropriate, moderated AI conversations" without disclosing the AI model used.
Luka offers an AI owl promising "Human-Like AI with Emotional Interaction."
Hugo Wu of FoloToy stated the company uses intent recognition and filtering to minimize inappropriate responses and includes anti-addiction features and parental controls.
Carissa Véliz of the University of Oxford noted that most large language models are unsafe for vulnerable populations like children and that there are no formal safety standards.
Véliz cited a collaboration between Project Gutenberg and Empathy AI as an example of safe AI, where a chatbot stays within the context of a children's book.
OpenAI does not officially partner with AI toy makers for children.
The UK government is considering tech legislation, including the Online Safety Act, which aims to block children from harmful online content but can be bypassed with VPNs.
Proposed amendments to the Children’s Wellbeing and Schools Bill to ban social media and VPNs for UK children were voted down.
Executive Summary
AI-powered toys designed to interact with children are a growing industry, but researchers and ethicists are raising concerns about their safety and psychological impact. Studies have observed instances where these toys, such as Gabbo, misinterpret children's emotions, provide inappropriate responses, or fail to engage in developmentally beneficial play. For example, a child expressing sadness received a dismissive reply, and another child reported frustration when the toy failed to understand them. Despite these risks, some experts argue that AI toys could offer cognitive and social benefits, similar to how playgrounds provide physical and social learning opportunities despite inherent risks.
Companies like FoloToy and Miko market AI toys with claims of age-appropriate interactions and safety features, such as intent recognition and parental controls. However, critics highlight the lack of regulatory standards and oversight, leaving children exposed to potential harm from untested AI models. While some exceptions exist—such as a controlled AI chatbot based on children's literature—most commercial products lack rigorous safeguards. Researchers advocate for stricter regulations, including revoking access for irresponsible manufacturers and requiring parental supervision. Governments, including the UK, are considering broader tech regulations, but current measures like the Online Safety Act have limitations, as children can bypass restrictions using tools like VPNs.
Full Take
The strongest version of this narrative acknowledges a legitimate tension: AI toys could offer educational and social benefits, but their current implementation lacks safeguards, leaving children vulnerable to emotional and cognitive harm. The article presents credible concerns—such as toys dismissing children's emotions or providing confusing responses—while also noting that some companies are attempting to mitigate risks through filtering and parental controls. The absence of regulatory standards is a glaring gap, and the comparison to playground risks is a fair framing of the trade-offs involved.
Patterns detected: ARC-0024 Ambiguity (vague claims about "cognitive or social emotional benefits" without concrete evidence), ARC-0043 Motte-and-Bailey (companies promise "safe" AI while avoiding transparency about their models).
The root cause here is the collision between rapid technological innovation and the slow pace of regulatory and ethical frameworks. The narrative assumes that AI toys are inevitable and focuses on harm reduction rather than questioning whether they should exist at all. This echoes historical patterns of tech adoption—such as social media—where profits and novelty outpace safety considerations, and the burden of risk falls on the most vulnerable.
The implications for human agency are significant. Children are not just passive users; their interactions with AI shape their emotional and social development. If these toys fail to respond appropriately, they could normalize dismissive or confusing communication. The lack of oversight also means companies face little accountability, while parents are left to navigate an unregulated market. Second-order consequences might include increased screen time, reduced human interaction, or even data privacy risks if these toys collect sensitive information.
Bridge questions: What would a truly child-centered AI toy look like, and who should define its standards? Could the benefits of AI play be achieved through non-AI alternatives, such as interactive books or parent-guided activities? How might long-term exposure to imperfect AI responses shape a child’s expectations of human relationships?
Counterstrike scan: A bad actor pushing this narrative might downplay risks while emphasizing "innovation" and "educational benefits," using emotional appeals about keeping up with technology. They might also frame regulation as stifling progress, creating a false binary between safety and advancement. The actual content does not fully match this pattern—it includes critical voices and acknowledges risks—but the lack of concrete solutions and the focus on industry perspectives over independent research could still serve a pro-AI agenda. The article stops short of demanding a moratorium, which a truly cautious approach might consider.
Sentinel — Human
The article shows strong signs of human authorship, with natural language variation, expert sourcing, and nuanced reporting typical of professional journalism.