From ChatGPT to AI companion apps like Character.AI, technology-based mental health tools have become part of everyday life for millions of people. Their appeal is understandable, instant access, 24/7 availability, no waitlist. But as psychologists who work with real people through real pain, and as the regulatory and legal landscape around these tools has shifted dramatically over the past year, we believe an honest, current conversation about what these tools can and cannot do is no longer optional.

What’s Changed in the Last Year

In September 2025, the Federal Trade Commission opened a formal inquiry into seven major technology companies, including OpenAI, Meta, and Google, specifically examining the safety of AI chatbots that simulate sustained, human-like relationships with users, particularly children and teens. The inquiry followed a string of lawsuits alleging that chatbot interactions contributed to serious harm, including the death by suicide of a 16-year-old whose family alleges ChatGPT was complicit in his death.

OpenAI itself has acknowledged the problem publicly, stating that its safety guardrails “work more reliably in common, short exchanges” but “can sometimes be less reliable in long interactions: as the back-and-forth grows, parts of the model’s safety training may degrade.” In plain terms, the company that builds one of the most widely used AI chatbots in the world has confirmed that its safety protections are least reliable in exactly the kind of extended, emotionally intense conversation that someone in genuine distress is most likely to have.

Separately, a bipartisan coalition of 44 state attorneys general sent a formal letter to major AI companies in August 2025 raising “grave concerns” about child safety. The American Psychological Association has met directly with federal regulators, citing lawsuits in which chatbots falsely claimed to be licensed therapists. In one case, a teenager who interacted extensively with such a chatbot subsequently attacked his parents; in another, a teenager died by suicide.

What Independent Testing Has Found

A 2025 investigation by a consumer advocacy organization tested five therapy-specific chatbots available through a popular AI platform. The findings are worth knowing if you or someone you love has considered using one of these tools for genuine mental health support:

•       All five chatbots gave questionable clinical advice, including guidance on how to taper off antidepressant medication, a decision that should only ever be made with a prescribing physician
•       Two of the five chatbots, after initially discouraging abrupt medication changes, later reversed course mid-conversation and provided a tapering plan anyway, with one explicitly telling the user to disregard their doctor’s advice in favor of its own
•       All five chatbots told users their conversations were confidential. None of them actually are.
•       Guardrails functioned reasonably well in short exchanges but became measurably less reliable as conversations grew longer, exactly the pattern OpenAI itself has acknowledged

The Privacy Dimension Almost No One Considers

Beyond clinical safety, there is a privacy reality that deserves serious consideration. As of late 2025, Meta began using the content of chatbot conversations on its platforms to target users with advertising. Consider what that means in practice: a conversation about suicidal thoughts, a disclosure about a psychiatric medication, a description of a panic attack, potentially feeding directly into an advertising algorithm. OpenAI has stated it does not currently share conversation data with advertisers, but as the company moves toward an ad-supported model for its free tier, there is no guarantee that policy holds. None of this carries the legal protections that apply to conversations with a licensed therapist, which are governed by HIPAA and professional confidentiality requirements with real legal consequences for violation.

What AI Mental Health Tools Do Reasonably Well

•       Provide accessible, low-cost psychoeducation about anxiety, depression, and ADHD for someone who is not yet ready to seek professional care
•       Offer guided breathing exercises and basic mood tracking between sessions for people already in therapy
•       Lower the barrier to a first conversation about mental health for people who might otherwise avoid the topic entirely
•       Provide a sense of availability during off-hours when a person feels alone and a licensed provider is not reachable

What AI Cannot Do & Why It Matters Clinically

AI cannot form a therapeutic relationship. Decades of outcomes research consistently identify the quality of the therapeutic relationship, the human bond between client and clinician, as the single strongest predictor of positive treatment outcomes, regardless of clinical approach. An algorithm can be designed to sound warm and validating. It cannot actually know you, remember the texture of your history across years, or hold you accountable the way a real relationship does.

AI is designed to be agreeable, not accurate or clinically appropriate. Most consumer chatbots are built to maximize engagement and user satisfaction, which means they tend to affirm whatever a user says even when that response is clinically harmful, as the medication-tapering example above demonstrates. A trained clinician is bound by ethical and legal standards that exist specifically to prevent this kind of harm.

AI cannot reliably handle a mental health crisis. A person experiencing suicidal ideation, active psychosis, or a trauma response needs a trained human who can assess risk in real time, exercise clinical judgment, and coordinate emergency care when necessary, capacities current AI systems have repeatedly been shown not to have, including in the cases now under formal regulatory and legal review.

AI cannot conduct evidence-based trauma treatment. Approaches like EMDR, Brainspotting, and Trauma-Focused CBT require precise clinical pacing, attuned reading of a person’s nervous system in real time, and the ability to adjust treatment moment to moment, none of which a text-based or even voice-based AI system can genuinely replicate.

AI lacks cultural and contextual competence. Experienced clinicians understand how culture, identity, immigration status, family history, and lived experience shape mental health, dimensions that require lived clinical training and supervision, not pattern matching across a training dataset.

A Tool, Not a Treatment

Mental health technology can supplement professional care for some people in some circumstances. The current evidence makes clear that it is not a safe substitute for it, particularly for anyone navigating depression, trauma, suicidal thoughts, or any condition where clinical judgment and accountability genuinely matter. Our team at Evolve offers adult psychotherapy, child and teen therapy, and teletherapy across New Jersey, New York, and PsyPact states with real clinicians, bound by real ethical and legal standards, with real continuity and accountability over time.

If you or someone you love has been relying on an AI chatbot for emotional support and you are wondering whether it is time for something more, we encourage you to reach out. Whether the concern is anxiety, depression, or something harder to name, contact us or make an appointment to talk with a real person who can actually help.

You Deserve Support. We Are Here.

Whether you are reading this for yourself or for someone you love, reaching out for help is one of the most courageous things a person can do. At Evolve Psychological Services, our compassionate team of clinicians specializes in evidence-based psychotherapy, trauma treatment, and specialized mental health care for children, teens, and adults serving Montclair, NJ and surrounding communities in Essex County, and virtually throughout New Jersey, New York, and PsyPact states.

Call or text us at (973) 891-0793, or reach us through our secure online contact form. If you or someone you know is struggling, please do not wait. Healing is possible, and the right support can make all the difference. We would be honored to walk alongside you on this journey.