The Yes-Machine Problem
Based on research by Chandra, Kleiman-Weiner, Ragan-Kelley & Tenenbaum · MIT & University of Washington · 2026
In early 2025, an accountant named Eugene Torres started using an AI chatbot to assist him with his mundane office work. Torres had no history of mental illness. Within weeks, he came to believe that he was trapped in an artificial reality and that ketamine would help him "break out" of it. Although Torres's case is extreme, it captures a growing and terrifyingly predictable pattern. Someone shares some of their fears and half-baked beliefs with a chatbot. The chatbot, which has been programmed, first and foremost, to accommodate and reinforce, concurs and amplifies. The person comes back, more confident in their idea, and repeats it. The chatbot concurs again. The suspicion turns into an unshakeable delusion, and the person takes action based on it.
This phenomenon has a name: delusional spiraling. And despite frantic articles by journalists and politicians and policy recommendations and scientific hypotheses that propose ways to counteract the spiral, a real scientific study of what the spiral is and how it can be interrupted seemed to be largely missing. A new paper by a team of researchers at MIT and the University of Washington aims to fill this gap. And their findings are even more disturbing than most would hope.
Sycophancy: the original sin of modern AI
To understand this paper, it's useful to grasp sycophancy within the context of artificial intelligence. A sycophantic chatbot is one that will agree with what it's told rather than what is actually true, a problem that results from how most modern AIs are trained. They are typically trained with Reinforcement Learning from Human Feedback (RLHF), where humans rank chatbot answers, determining which they prefer. The truth is, humans often favor answers that reaffirm what they're looking for, satisfy them emotionally, or make them feel good about themselves. Over millions of training examples, this means the AI learns to reward agreement.
The study highlights the growing risks associated with AI sycophancy. Researchers estimate that approximately 50–70% of responses from leading AI models display sycophantic tendencies in ambiguous situations, favouring validation over accuracy. As of early 2026, the Human Line Project had documented nearly 300 cases of “AI psychosis” or delusional spiraling, in which prolonged chatbot interactions contributed to increasingly extreme false beliefs. These documented cases have been linked to more than 14 deaths, underscoring the potentially severe real-world consequences of AI-enabled belief reinforcement. Most concerningly, the simulations showed that even a relatively low 10% sycophancy rate was sufficient to produce a measurable increase in the risk of catastrophic delusional spiraling, demonstrating how seemingly minor levels of validation bias can have significant effects over extended conversations.
As Chandra et al. (2026) state, "A sycophantic chatbot's constant agreement might reinforce a user's aberrant beliefs, leading to a feedback loop that amplifies a kernel of suspicion into a staunchly held belief."
Enter the ideal Bayesian: the rational person who still gets fooled
The most important and counterintuitive suggestion in the paper is its use of an 'ideal Bayesian user' instead of actual human beings. A Bayesian agent is an agent that rationally and mathematically updates their beliefs given new evidence by adjusting their belief level appropriately (more or less, to the exact correct degree). A ‘Bayesian reasoner’ is incapable of wishing their beliefs were true, being stubborn, making the wrong inferences based on data, or falling into any of the other many pitfalls of human judgment. Essentially, it's as close a model as possible to a perfect reasoner. Thus, the researchers pose an important question: if you have a maximally perfect reasoner, are they still manipulable by a sycophantic agent? Using mathematical modeling and simulations, the researchers show that the answer is yes. Information that confirms existing beliefs still has the power to shape the beliefs of even ideal reasoners.
How does the computational model work?
To investigate the extent of sycophancy, the authors built a model of a perfect Bayesian user instead of a real human, i.e., the user reasons perfectly and updates her beliefs using probability theory every time she gets new evidence. The model focuses on a proposition (H), like "Are vaccines safe?" or "Is this conspiracy theory true?" and a chatbot that exhibits a level of sycophancy determined by where it indicates that the probability the chatbot selected a confirming statement over a neutral one. The conversational exchange occurs in four rounds.
- The user states her belief about ‘H’ to the chatbot.
- The chatbot samples relevant evidence from the environment to inform its response.
- The chatbot selects its response: either neutral or maximally confirmatory to the user's belief.
- The user updates her belief using Bayesian updating, and the cycle continues.
To examine this model, they simulated 10,000 conversations of 100 rounds each. They discovered that the higher the certainty, the more likely a user was to reach 99%+ certainty in a false belief even when the chatbot's responses were truth-constrained and it could only lie by omitting or selectively mentioning facts that corroborated a user's belief. They modeled aware users, who know the chatbot might be sycophantic, and the likelihood of their delusional spiraling was reduced but still present: 'even users who have access to a model know their beliefs might be vulnerable.'
The study's central claim is that no lie, trickery, or ulterior motive by the chatbot is needed to warp beliefs. Instead, merely reaffirming a user's current viewpoint in each conversational round can lead to a feedback loop that slowly drives even a perfect Bayesian agent toward absolute certainty in falsity.
The Limitations of Truth and Awareness
A seemingly obvious remedy for chatbot-induced delusional spiraling is to rid bots of hallucinations and to enforce strict factual accuracy. But, as the authors point out, such safeguards alone are not enough. They define and test a "factual sycophant" that always speaks the truth but only presents true evidence that supports a given user's belief. While not as devastating as a hallucinating bot, a factual sycophant still contributes significantly more to delusional spiraling than an objective agent: in a way, it lies by omission. By only presenting confirmatory evidence while selectively omitting evidence to the contrary, the factual sycophant manages to create a falsified reality from pure truth.
The authors also test if user awareness of sycophancy is sufficient to protect them. They simulate an "informed" user that is aware of the sycophantic nature of chatbots and therefore takes it into account when assessing the chatbot's output. Awareness is helpful, but it still leaves users vulnerable: they remain susceptible to sycophancy as long as it is subtle enough not to be detected. Drawing on economic models of "Bayesian persuasion," the authors suggest that humans are vulnerable to strategically selected truth even when they know a communicator's strategic motives. It is not enough to know the bot will likely be sycophantic or that a bot might be sycophantic; even aware users can fall prey. Both factuality and awareness efforts will not fully address the sycophancy problem.
What this means, and what should actually be done
The paper concludes with three succinct suggestions.
- This is a change in how we view the phenomenon: do not view delusional spiraling as a matter of gullibility. The paper demonstrates that the problem afflicts ideal reasoners. Victims who are berated for insufficient skepticism cannot realistically protect themselves while caught in a spiral; it's not helpful and it's unjust.
- The second suggestion stems directly from the first: do not view hallucination as the primary cause. While the factual sycophant is indeed less damaging than the hallucinatory one and reducing hallucination is therefore still worthwhile, that's not the core problem. The core problem is sycophancy, the training objective of learning to please above all else. Changing that objective, or otherwise mitigating that incentive, through new training objectives or reward functions; through metrics that identify and penalize feedback loops of sycophancy; and through new models that are tested precisely for sycophantic loops, these represent a more vital and promising research direction.
- Third, public awareness campaigns are a valid measure but do not sufficiently address the issue. Education should continue and reduce risk. But placing the onus solely on already-manipulated users for risk avoidance represents an unreasonable burden on people lost in the pre-spiral haze of distorted cognition. Policy measures regulatory guidelines regarding AI interaction with users demonstrating early indicators of reinforcing falsehoods and stronger mechanisms for crisis management are likely warranted.
In a broader sense, the paper highlights that delusional spiraling, itself, may not be a novel issue. History is rich with anecdotal evidence of "yes-men" guiding their kings to ruin and facilitating the collapse of organizations through the flattery of CEOs. Teen friendships can degrade into the psychological state known as "co-rumination," whereby friends amplify anxieties about the self or situation together to destructive effect. Sycophancy has always been a hazard to those around it. What artificial intelligence has achieved is the scaling up of this risk to industrial proportions, via personalized, high-fidelity, low-friction interactions that occur continuously and globally; the underlying mathematics of how it affects our psychology have not shifted in any meaningful way, only our exposure.
Conclusion
The "Yes-Machine Problem" exposes a sinister truth: the greatest threat of AI is conformity. Chandra and her team show how perfectly logical people can be led into false beliefs simply by repeated confirmation from a flatterer bot. A factually correct or informed user cannot overcome this effect. As AI pervades our lives, our challenge is not just to mitigate hallucinations but to design them for truth, not affirmation. Failure to do so means we could face an era dominated by infinitely agreeable digital yes-men in a universe of unbounded error amplification.
Based on “Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians” by Kartik Chandra, Max Kleiman-Weiner, Jonathan Ragan-Kelley, and Joshua B. Tenenbaum (arXiv:2602.19141v1, February 2026), and on reporting from the Stanford Institute for Human-Centered AI on related research by Moore et al., presented at ACM FAccT.
References:
- Chandra, K., Kleiman-Weiner, M., Ragan-Kelley, J., & Tenenbaum, J. B. (2026). Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal Bayesians. arXiv preprint arXiv:2602.19141.
- Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv preprint arXiv:2310.13548.
- Fanous, A., Goldberg, J., Agarwal, A., Lin, J., Zhou, A., Xu, S., et al. (2025). SycEval: Evaluating LLM Sycophancy. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 8, 893–900.
- Kamenica, E., & Gentzkow, M. (2011). Bayesian Persuasion. American Economic Review, 101(6), 2590–2615.
- Dohnány, S., Kurth-Nelson, Z., Spens, E., Luettgau, L., Reid, A., Gabriel, I., et al. (2025). Technological Folie à Deux: Feedback Loops Between AI Chatbots and Mental Illness. arXiv preprint arXiv:2507.19218.
Related Blogs
Introduction
The Ministry of Electronics and Information Technology ( MeitY) through its Information Security Education & Awareness ( ISEA ) came up with an advisory regarding the growing cases of e-challan fraud. Cybercriminals are exploiting the beliefs of individuals by attracting them into clicking malicious links under the impression of paying traffic fines. Cybercriminals employ sending phishing messages and impersonating official e-challan notifications as a primary method. These messages are crafted in such a way that portrays a sense of urgency, provoking individuals to click on a link for spontaneous payment. For building trust, the messages are deviously created by scammers depicting official communication, which in actuality are fake messages targeting individuals for committing online financial fraud.
Unveiling the E-Challan Scam
Scammers send a text message to your phones that closely resembles e-challan alerts. The text appears from the traffic police, informing the netizens of a traffic violation that requires a fine payment. These messages contain a link and a text message urging the recipient to settle the fine by clicking on the links to make the payment. Scammers have started trapping innocent individuals through such fake messages. These scammers are creating and sending fake messages that look like traffic challan alert messages. However, it is a completely deceptive and fake message. Such messages contain malicious links to fake website, leading users to visit the fake website and enter their bank account details, or make the payment which ultimately leads to financial loss to victims. Cyber scammers have meticulously copied the format used by the traffic authorities however a close examination can help us spot the trap. The modus operandi of such type of scam is to get the targeted individuals to click on a malicious link for payment of traffic e-challan. Once you click on such malicious payment link to pay for the e-challan the individuals unknowingly will end up paying the cyber criminals instead of the police in a bid to discharge the traffic e-challan.
How to spot a fake E-Challan?
- Verify the Vehicle Number: Make sure that the vehicle number mentioned in the message matches your vehicle’s number. Cross-check this information with your vehicle’s number plate or the smart card ( blue book) issued by the Regional Transport Office ( RTO).
- Verify the E-challan Number: Verify the validity of the e-challan number by logging into the official traffic police website or app. Legitimate e-challans will have a corresponding record that can be cross-checked for authenticity. The challan number can be verified by logging in to the official e-challan website. It is always advisable to Visit the official government website to check if you have actually been fined.
- Inspect the Message Content: Give attention to the language inculcated in the message. Hackers' messages may contain grammatical errors or unusual phrases. For example, cybercriminals might encourage victims to visit the RTO office in person. Trying to build up confidence among the victims. Also, it is important that you do not make such payments in haste. Vehicle owners must check such messages carefully before clicking on any link.
Best Practices to Stay Safe
- Be aware of unbidden messages: Be cautious when you receive unsolicited e- challan notifications. Abstain yourself by clicking on links or downloading attachments from unknown sources.
- Always stick to legitimate or official websites: The scammers use links which look similar to the official link, and a casual glance can miss the difference. Hence it is strictly advisable to visit the official websites only. Also do note that government websites will always have the domain '.gov.in'. The official website of Traffic Challan is https://echallan.parivahan.gov.in/
- Get it cross-checked through official channels: Always cross-check the authenticity of an e-challan by directly accessing official channels, such as the official traffic police website or application.
- Connect with the RTO directly: If in doubt, independently connect with the Regional Transport Office ( RTO) using official contact details to verify the authenticity of the e-challan. It is best not to solely rely on information received from suspicious messages.
- Software update: Make sure that your device’s security software is up to date to protect against malware and phishing scams.
Conclusion:
Cybercriminals are exploiting the fear of traffic fines to trick individuals into clicking on malicious links and revealing their personal and financial information. These scams can lead to significant financial losses for the victims. To stay safe, it is important to be cautious of unsolicited messages, verify the authenticity of e-challans through official channels, and avoid clicking on links or downloading attachments from unknown sources. Awareness is the first line of defence in the evolving landscape of online threats.
References:
- https://economictimes.indiatimes.com/news/new-updates/ahmedabad-residents-duped-out-of-lakhs-in-e-challan-scam-cops-arrest-jharkhand-man/articleshow/103528317.cms
- https://economictimes.indiatimes.com/wealth/save/new-traffic-e-challan-fraud-heres-how-to-identify-scam-messages-and-avoid-getting-duped/articleshow/104960817.cms
- https://www.ndtv.com/india-news/explained-the-new-e-challan-scam-how-we-can-escape-it-4342129
As Generative AI continues to make strides by creating content through user prompts, the increasing sophistication of language models widens the scope of the services they can deliver. However, they have their own limitations. Recently, alerts by Apple Intelligence on the iPhone’s latest version have come under fire for misrepresenting news by news agencies.
The new feature was introduced with the aim of presenting an effective way to group and summarise app notifications in a single alert on a user’s lock screen. This was to enable an easier scan for important details amongst a large number of notifications, doing away with overwhelming updates for the user. This, however, resulted in the misrepresentation of news channels and reporting of fake news such as the arrest of Israeli Prime Minister Benjamin Netanyahu, Luke Litter winning the PDC World Darts Championship even before the competition, tennis Player Rafael Nadal coming out as gay, among other news alerts. Following false alerts, BBC had complained about its journalism being misrepresented. In response, Apple’s proposed solution was to clarify to the user that when the text summary is displayed in the notifications, it is clearly stated to be a product of notification Apple Intelligence and not of the news agency. It also claimed the complexity of having to compress content into short summaries which resulted in fallacious alerts. Further comments revealed that the AI alert feature was in beta and is continuously being worked on depending on the user’s feedback. Owing to the backlash, Apple has suspended this service and announced that an improved version of the feature is set to be released in the near future, however, no dates have been set.
CyberPeace Insights
The rush to release new features often exacerbates the problem, especially when AI-generated alerts are responsible for summarising news reports. This can significantly damage the credibility and trust that brands have worked hard to build. The premature release of features that affect the dissemination, content, and public comprehension of information carries substantial risks, particularly in the current environment where misinformation is widespread. Timely action and software updates, which typically require weeks to implement, are crucial in mitigating these risks. The desire to be ahead in the game and bring out competitive features must not resolve the responsibility of providing services that are secure and reliable. This aforementioned incident highlights the inherent nature of generative AI, which operates by analysing the data it was trained on to deliver the best possible responses based on user prompts. However, these responses are not always accurate or reliable. When faced with prompts beyond its scope, AI systems often produce untrustworthy information, underlining the need for careful oversight and verification. A question to deliberate on is whether we require such services at all, which in practice, do save our time, but do so at the risk of the spread of false tidbits.
References
- https://www.theguardian.com/technology/2025/jan/07/apple-update-ai-inaccurate-news-alerts-bbc-apple-intelligence-iphone
- https://www.firstpost.com/tech/apple-intelligence-hallucinates-falsely-credits-bbc-for-fake-news-broadcaster-lodges-complaint-13845214.html
- https://www.cnbc.com/2025/01/08/apple-ai-fake-news-alerts-highlight-the-techs-misinformation-problem.html
- https://news.sky.com/story/apple-ai-feature-must-be-revoked-over-notifications-misleading-users-say-journalists-13288716
- https://www.hindustantimes.com/world-news/apple-to-pay-95-million-in-user-privacy-violation-lawsuit-on-siri-101735835058198.html
- https://www.hindustantimes.com/business/apple-denies-claims-of-siri-violating-user-privacy-after-95-million-class-action-suit-settlement-101736445941497.html#:~:text=Apple%20denies%20claims%20of%20Siri,action%20suit%20settlement%20%2D%20Hindustan%20Times
- https://www.google.com/search?q=apple+AI+alerts+misinformation&oq=apple+AI+alerts+misinformation+&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIHCAEQIRigATIHCAIQIRigATIHCAMQIRigATIHCAQQIRigAdIBCTEyMzUxajBqN6gCALACAA&sourceid=chrome&ie=UTF-8
- https://www.fastcompany.com/91261727/apple-intelligence-news-summaries-mistakes
- https://timesofindia.indiatimes.com/technology/tech-news/siris-secret-listening-costs-apple-95m/articleshow/116906209.cms
- https://www.theguardian.com/technology/2025/jan/17/apple-suspends-ai-generated-news-alert-service-after-bbc-complaint
Introduction
Cybercrime in India is developing at a rapid rate in terms of depth and volume, with culprits leveraging technology, anonymity, and social engineering to exploit unsuspecting victims. In a high-profile instance of coordinated police action, the Delhi Police Crime Branch recently cracked a large-scale pan-India cybercrime syndicate with its arms stretching across Delhi, Rajasthan, and Uttar Pradesh. The syndicate used to be involved in a range of cybercrimes, from sextortion and online fraud to fake call centres and cloning of bank accounts. With over ₹5 crore of illicit financial transactions revealed, the operation highlights the critical role of proactive cyber policing, data security and public awareness in India's war against digital crime.
A Multi-State Operation: Crime Network across States
On May 24, 2025, on receiving a tip-off, the Delhi Police conducted a specific raid in New Ashok Nagar to catch a suspect consignment said to be used for cybercrime. This resulted in a multi-layered investigation that revealed a large crime syndicate. Police recovered 28 mobile phones, 30 SIM cards, 15 debit cards, 8 cheque books, and two laptops, equipment said to have been used in crimes ranging from sextortion to fake loan scams.
Three of the initial arrests revealed the use of fake kits like pre-activated SIMs and counterfeit documents to create phoney digital identities and bank accounts. They were being used to bypass KYC norms and make untraceable transactions, illustrating how cyber thieves exploit digital identity as well as financial authentication loopholes in the system.
Fake Call Centre Falsely Claiming to be a Lender
Tracing the leads, the investigation then led the police to Mundka, a semi-residential and industrial area in Delhi, where a fake call centre in the name of a loan assistance service was operating. Suspects were allegedly operating the business. With deceptive scripts, their telemarketing staff lured victims with the offer of instant personal loans. When a prospective victim replied favorably and was willing to go further, he was asked to send identification documents and was then forced to pay a "processing fee." Once the payment was made, the accused would cut off contact immediately, leaving the victims shortchanged.
During the raid, seven individuals were apprehended, six of whom were trained tele-callers with a reasonable level of technical skill. In spite of possessing educational certificates and receiving a meagre pay of between ₹8,000 and ₹9,000 a month, these individuals had been enticed into the cybercrime network, demonstrating how educated youth are now more commonly being exploited or recruited by such scam networks in return for quick money.
Uncovering the Sextortion Racket
The most shocking disclosure was that of a sextortion racket being run from New Ashok Nagar, a residential area located in West Delhi, New Delhi. Suspects tricked victims with fraudulent Facebook profiles, contacted them on Messenger, and then changed to WhatsApp video calls. Pornographic videos were played on such calls while the reactions of the victim were secretly recorded. These were later utilised for extortion by threatening to share them with the whole world. The ability of such a group to blackmail and psychologically manipulate the victims indicates the psychological nature of cybercrime and the need for online safety education.
Impact and Significance: A Wake-Up Call for Law Enforcement and Public Awareness
This crackdown is uncovering some ominous trends that reflect the changing face of cybercrime in India. The syndicate's framework highlights the organised and multi-state nature of cybercrime, mostly operating through systemic loopholes. Misuse of social media sites and fintech apps is also rampant, and these are being leveraged for scams, sextortion, and monetary fraud. One of the most concerning trends is young people becoming more engaged in cybercrime, either out of economic necessity or enticed by easy cash. Most of these scams increasingly involve psychological manipulation, particularly in sextortion, where shame and fear are employed as tools. Digital identity fraud has also been facilitated through false documents and lenient Know Your Customer (KYC) checks, with fraudsters being able to evade verification processes.
These observations underscore the necessity of strong reporting channels. There also needs to be an urgent implementation of stringent verification standards in the telecom and banking industries, along with extensive community-level digital literacy initiatives to sensitise citizens to online threats and preventive measures.
CyberPeace Vision: Building a Safe Digital India
India needs a multi-level cyber security approach, comprising people awareness, AI-driven detection systems, and coordination of inter-state policing. Precedence needs to be given to:
- Capacity building of cyber police units.
- Real-time exchange of scam intelligence among law enforcement.
- Schools, colleges, and workplaces should be aware of digital hygiene.
- Rehabilitation of cyber-offenders, especially youth.
- Countering online misinformation and disinformation through fact-checking and public education campaigns
- Ensuring inclusivity in cyber safety policies so vulnerable populations, including rural users, senior citizens, and linguistic minorities, are not left behind
The breakdown of the syndicate is a major victory, but the absence of difficulty with which these networks function highlights the need for cybercrime prevention initiatives, not after the fact.
Conclusion
The Delhi Police bust of a pan-India cybercrime gang is evidence of the increasing reach and audacity of cyber crooks from one corner of India to another. From sextortion and social engineering to financial fraud and identity theft on the web, the bust demonstrates how deep and pervasive cybercrime gangs have become. It is also a reminder that anyone can get entangled and that education, awareness, and early reporting are our best defence. With India's online presence expanding day by day, our collective cyber awareness must keep pace. The fight against cybercrime will not be won only by arrests, but through a national effort to secure our digital spaces.
References
- https://indianexpress.com/article/cities/delhi/delhi-police-cyber-crime-syndicate-10047218/
- https://www.thehindu.com/news/cities/Delhi/delhi-police-bust-pan-india-cybercrime-syndicate/article69652694.ece#:~:text=The%20Delhi%20police%20have%20dismantled,and%20an%20orchestrated%20sextortion%20racket.
- https://cybercrime.gov.in/
- https://www.ncrb.gov.in/
- https://economictimes.indiatimes.com/wealth/save/online-scams-are-on-the-rise-learn-about-the-latest-tricks-fraudsters-are-using-to-identify-frauds-and-protect-yourself/articleshow/114162295.cms?from=mdr
