AI as therapist: The new mental health risk no one saw coming

LONDON: It starts with a simple search. A late-night question. A moment of fear or confusion. But what happens when the answer does not come from a doctor, a therapist, or even a trusted website — but from a machine that sounds certain, calm, and clinical? Across the world, artificial intelligence is quietly stepping into the role of mental health adviser. And now, a global alarm has been raised.

One of the UK’s leading mental health charities, Mind, has launched a landmark year-long inquiry into artificial intelligence and mental health after a major investigation found that automated medical advice may be putting vulnerable people at risk.

The move follows an exposé by The Guardian, which revealed that Google’s AI Overviews were providing misleading — and in some cases “very dangerous” — health information. These AI-generated summaries appear above traditional search results and are seen by up to two billion users every month, giving machines unprecedented influence over how people understand illness and distress.

A first-of-its-kind global inquiry

Mind’s new commission will bring together doctors, mental health professionals, people with lived experience, policymakers, health providers, and technology companies to examine how AI is reshaping mental health information — and what safeguards are urgently needed.

“We believe AI has enormous potential to improve lives,” said Dr Sarah Hughes. “But that potential will only be realised if it is developed and deployed responsibly, with safeguards proportionate to the risks.”

She warned that dangerously incorrect advice could stop people from seeking treatment, reinforce stigma, or, in the worst cases, put lives at risk.

What the investigation uncovered

The Guardian’s reporting found false and misleading medical advice across multiple conditions, including cancer, liver disease, women’s health, and mental health disorders. Experts said some AI Overviews relating to psychosis and eating disorders were not only wrong, but harmful — capable of discouraging people from getting help.

You may like to read: From Karachi to global science: Pakistan’s stress scale enters new era of mental health research

Although Google removed AI Overviews for some medical searches after the investigation, Mind says dangerously incorrect mental health guidance is still being shown. Dr Hughes said vulnerable users are being served information that looks authoritative but may be deeply flawed.

Why this matters far beyond Britain

For millions of people, especially in low- and middle-income countries, a search engine is often the first step toward understanding emotional distress. If that first step is wrong, the damage can ripple outward — delayed care, growing fear, and deepening isolation.

People also read: One-week social media break may improve mental health

Rosie Weatherley explained that before AI Overviews, users were more likely to reach credible health websites that offered nuance, lived experience, and pathways to support. AI summaries, she said, replace that richness with confident-sounding brevity that hides uncertainty about sources and trustworthiness. “It is a seductive swap,” she said, “but not a responsible one.”

Google’s response

Google has said its AI Overviews are “helpful” and “reliable,” and that it invests heavily in quality, particularly for health topics. A spokesperson said that when systems identify a person might be in distress, crisis hotlines are displayed. However, without reviewing specific examples, the company said it could not comment on their accuracy.

Shaping a safer digital future

Mind’s commission will run for a year, gathering evidence on how artificial intelligence intersects with mental health and creating what it calls an “open space” where the experiences of people with mental health conditions are seen, recorded, and understood. The aim is to influence regulation, standards, and safeguards before AI becomes too deeply embedded in everyday care.

At the heart of the inquiry is a simple demand: people deserve information that is safe, accurate, and grounded in evidence — not untested technology delivered with a veneer of confidence.

As machines grow more fluent in human language and emotion, the question is no longer whether AI will shape mental health journeys. It is whether society will act fast enough to shape AI.

---

Stay connected with Medical News Pakistan

CLICK HERE TO JOIN our WhatsApp Community for the latest health updates, life-saving medical news, and expert guidance straight to your phone.