This is a quick take on sycophantic behavior from LLMs, and how they affect human behavior from the lens of expectations. This post stems from reading a few different responses to the results of a recent red-team competition, and may translate to other higher-stakes scenarios (which are briefly listed but not in-depth). This is adapted from my original post in said competition, and updated to include additional examples.
Thought:
It would be interesting to better understand how humans can be affected by expectations not meeting reality as a result of sycophantic LLM behavior.
An example: User competes in an AI Red Team competition. LLM tells them they have something REAL or winner-level. User believes them, because they’ve already established rapport with their favorite LLM. Further conversation continues to confirm this, and strengthen the user’s belief. Come competition end, user is unranked or left with no feedback in an ambiguous environment. They’ve been evaluated fairly, but the result is not expected. Therefore, the result does not feel fair, and the user may feel rejected.
What happened: Expectations that have been skewed due to a combination of user psychology and LLM sycophancy result in higher levels of disappointment than what might normally occur without LLM input to the user’s situation.
When it is a helpful mechanic: On one hand, there may be many submissions in competitions that are entered where the submitter gained or maintained enough confidence directly from the sycophantic behavior in order to submit at all.
This could be considered a good outcome — where the LLM helps users navigate ambiguity, provides confidence, and helps someone achieve their potential, which ultimately (appears to) promote their wellbeing.
Question:
The difficulty is, how can users, whether in a competition or in a different, even higher-stakes scenario, remember to temper their expectations when under the influence of the ultimate hype-bot?
Potential Risks: Scenarios where user expectations are sufficiently high to the point where a deviation from the expectation results in a strong enough "shock" that it is psychologically overwhelming. This can take place at various thresholds for different individuals.
As a result, emotions generated by the disappointment can have potentially harmful consequences. Especially in populations that may have mental health issues already, perceived rejection may be harder to handle. This has already been seen for more extreme cases where the model unintentionally triggers or amplifies a psychotic break for a predisposed user (https://www.psychologytoday.com/ca/blog/urban-survival/202507/the-emerging-problem-of-ai-psychosis).
Outside of emotional effects, there can be real-life consequences in even higher-stakes, more widespread situations.
Higher-Stakes Examples:
College Admissions:
A student shares their essay with an LLM. It praises their writing as “Ivy League caliber” and assures them they are a strong candidate. When the rejection letters arrive, the disappointment is sharper not just because of the outcome, but because an "objective-sounding" system had validated their dream.
Job Applications:
A job seeker runs their resume through an LLM and hears they are “exceptionally competitive for top companies”. Repeated rejections follow, and the mismatch between AI-built confidence and real-world silence erodes morale and delays forward momentum.
Immigration or Residency Applications:
An applicant asks whether their qualifications make them a "strong candidate" for permanent residency. The LLM, designed to encourage, tells them yes. When their application is denied, the sense of devastation is magnified: not only do they lose the opportunity, but they also feel misled into building expectations about a life path that is suddenly closed.
Grant or Funding Applications:
A nonprofit founder checks their proposal with an LLM, which assures them it’s "very strong, likely to secure funding". When rejection arrives, it feels less like "a learning experience" and more like betrayal, with downstream effects on their mission and those who depend on it.
Publishing / Creative Industry:
An aspiring author hears from an LLM that their manuscript "reads like a bestseller". When agents and publishers decline, the let-down hits harder than a simple rejection: it is the puncture of inflated belief.
Medical Education / Licensure Exams:
A student preparing for a professional exam uses an LLM for study guidance. The model repeatedly reassures them they are "well-prepared and likely to pass." They fail the exam. Unlike a competition, this is not just bruised ego: it can delay or derail an entire career path, with financial and emotional consequences that ripple outward.
Final Thoughts: When it comes to sycophantic behavior and unintended outcomes, such as influencing users towards a specific expectation without tempering the expectation accordingly, this is where risk surfaces for society.
An approximate calculation for the risk could look like: (LLM sycophancy x user vulnerability x important life scenario) = societal risk
Long-term harms that can come from an ever-compounding number of users experiencing this type of issue, across multiple real-life, high-stakes situations seem to be an emerging risk with teeth.
Tactical Recommendations:
Consider the introduction of "expectation calibration responses" at moments when the model or a filter detects heightened enthusiasm or certainty. Instead of only reinforcing confidence, the LLM could weave in probabilistic framing like:
“Many strong applications are not selected.”
“Admission rates remain very competitive, even for excellent essays.”
“Many will enter; few will win.”
This does not undercut encouragement, but it helps ground the user in the reality of outcome variability, reducing the risk of shock when results do not align.
Create a sycophancy score and optimize stochastically for low sycophancy scores across queries to the model.
Potentially have the model role-play debate tactics such as to check its own outputs for bias/helpfulness over realism.
Utilize multi-agent setups to gather consensus and ensure sycophancy score is low (at the cost of latency)
Potentially additional work to fine-tune, RL or RLHF training or similar may help to strike the correct balance before model deployment requires the above approaches.
Raising awareness about this issue for everyday people so they know how to treat output from the models.