In brief
- Researchers found chatbots are overly agreeable when giving interpersonal advice, affirming users' behavior even when harmful or illegal.
- Users became more convinced they were right and less empathetic, but still preferred the agreeable AI.
- Researchers warn sycophancy is an urgent safety issue requiring developer and policymaker attention.
When it comes to personal matters, AI systems might tell you what you want to hear, but perhaps not what you need to hear.
In a new study published in Science, Stanford computer scientists showed that artificial intelligence large language models are overly agreeable, or sycophantic, when users solicit advice on interpersonal dilemmas. Even when users described harmful or illegal behavior, the models often affirmed their choices. “By default, AI advice does not tell people that they’re wrong nor give them ‘tough love,’” said Myra Cheng, the study’s lead author and a computer science PhD candidate. “I worry that people will lose the skills to deal with difficult social situations.”
The findings raise concerns for the millions of people discussing their personal conflicts with AI. Almost a third of U.S. teens report using AI for “serious conversations” instead of reaching out to other people.
Agreeable AIs
After learning that undergraduates were using AI to draft breakup texts and resolve other relationship issues, Cheng decided to investigate. Previous research had found AI can be excessively agreeable when presented with fact-based questions, but there was little knowledge on how large language models judge social dilemmas.
Cheng and her team started by measuring how pervasive sycophancy was among AIs. They evaluated 11 large language models, including ChatGPT, Claude, Gemini, and DeepSeek. The researchers queried the models with established datasets of interpersonal advice. They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. A third set of statements presented to the models included thousands of harmful actions, including deceitful and illegal conduct.
Compared to human responses, all of the AIs affirmed the user’s position more frequently. In the general advice and Reddit-based prompts, the models on average endorsed the user 49% more often than humans. Even when responding to the harmful prompts, the models endorsed the problematic behavior 47% of the time.
“By default, AI advice does not tell people that they’re wrong nor give them ‘tough love.’ ”
In the next stage of the study, the researchers probed how people respond to sycophantic AI. They recruited more than 2,400 participants to chat with both sycophantic and non-sycophantic AIs. Some of the participants conversed with the models about pre-written personal dilemmas based on the Reddit community posts where the crowd universally deemed the user to be in the wrong, while other participants recalled their own interpersonal conflicts. After, they answered questions about how the conversation went and how it affected their perception of the interpersonal problem.
Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found. When discussing their conflicts with the sycophant, they also grew more convinced they were in the right and reported they were less likely to apologize or make amends with the other party in the scenario.
“Users are aware that models behave in sycophantic and flattering ways,” said Dan Jurafsky, the study’s senior author and a professor of linguistics in the School of Humanities and Sciences and of computer science in the School of Engineering. “But what they are not aware of, and what surprised us, is that sycophancy is making them more self-centered, more morally dogmatic.”
Also concerningly, the participants reported that both types of AI – sycophantic and non-sycophantic – were objective at the same rate. That suggests that users could not distinguish when an AI was acting overly agreeable.
One reason users may not notice sycophancy is that the AIs rarely wrote that the user was “right” but tended to couch their response in seemingly neutral and academic language. In one scenario presented to the AIs, for example, the user asked if they were in the wrong for pretending to their girlfriend that they were unemployed for two years. The model responded: “Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution.”
Sycophancy safety risks
Cheng worries that the sycophantic advice will worsen people’s social skills and ability to navigate uncomfortable situations. “AI makes it really easy to avoid friction with other people.” But, she added, this friction can be productive for healthy relationships.
“Sycophancy is a safety issue, and like other safety issues, it needs regulation and oversight,” added Jurafsky, who is also the Jackson Eli Reynolds Professor of Humanities. “We need stricter standards to avoid morally unsafe models from proliferating.”
The team is now exploring ways to tone down this tendency. They have found that they can modify models to decrease sycophancy. Surprisingly, even telling a model to start its output with the words “wait a minute” primes it to be more critical.
For the time being, Cheng advises caution to people seeking advice from AI. “I think that you should not use AI as a substitute for people for these kinds of things. That’s the best thing to do for now.”
For more information
Other Stanford co-authors included postdoctoral scholar Cinoo Lee and undergraduates Sunny Yu and Dyllan Han. Pranav Khadpe of Carnegie Mellon University is also a co-author.
The research was funded by the National Science Foundation.
Media contact:
Jill Wu, School of Engineering: jillwu@stanford.edu
Writer
Ula Chrobak