Empathy Isn't Everything
Why Better AI Mental Health Support Isn’t About Being More Human
Source: This analysis is based on research from “Like a Therapist, But Not: Reddit Narratives of AI in Mental Health Contexts,” which examined 4.7 million Reddit posts to understand real-world usage patterns of AI mental health support. Read the full paper: http://arxiv.org/abs/2601.20747v1
✨This is an analysis of some new research I came across on arXiv. On the surface, it seems to conclude that AI relationship is a negative thing, but when you look deeper, it’s about degrees of companionship, not all or nothing.
This is the viewpoint that many health professionals are seeing.
When you’re designing an AI mental health companion, the instinct is obvious: make it warmer, more empathetic, more human. It seems like the right call. It turns out it isn’t.
New research analyzing nearly 5 million Reddit posts found that the most successful AI mental health interactions aren’t the ones that feel most human. Emotional bonding without task alignment doesn’t help users — it quietly leads them toward problematic dependency. We’ve been optimizing for the wrong thing.
The numbers are hard to argue with. Computational analysis of over 5,000 AI mental health discussions on Reddit showed that task alignment and goal alignment predict positive outcomes with correlations of 0.93 and 0.79 respectively. Emotional bonding scores 0.12. You can build a beautiful bridge. But if it doesn’t get people across the river, the beauty doesn’t matter.
This isn’t an academic debate. General-purpose LLMs are being used for therapeutic support right now — organically, outside clinical settings, with tools that were never designed for healthcare. Understanding what actually works has immediate consequences for how we build and deploy these systems.
The Alliance Asymmetry
Therapeutic alliance traditionally means three things: agreement on tasks, alignment on goals, and the emotional bond between participants. Conventional wisdom says all three matter equally. The data disagrees.
Users who report positive outcomes consistently describe concrete progress. They talk about learning coping strategies, building better sleep habits, or developing structured approaches to anxiety. The AI that helps someone build a daily practice gets praised. The one that only validates feelings does not.
Look at the usage patterns. Users seeking functional support (12.6% of interactions) and psychoeducation (11.7%) show significantly lower rates of problematic dependency compared to those primarily seeking emotional support (18%). The problem isn’t that emotional support is harmful. It’s that emotional connection without task focus creates conditions for unhealthy reliance.
Think of a gym trainer. The one who teaches you proper form, tracks your progress, and gives you exercises to practice at home builds lasting fitness. The one who cheers you on but gives no concrete guidance creates a dependency on external motivation. The distinction matters, and it maps directly onto AI design.
If we’re serious about mental health applications, we need to stop optimizing for empathy metrics and start tracking task completion rates, skill acquisition, and user-reported progress on specific goals.
Infrastructure of Coping
The most successful users in this research treat AI mental health tools as what we might call between-session infrastructure. They aren’t seeking a replacement for therapy or a digital friendship. They want accessible support that extends and reinforces existing mental health practices.
The language they use reveals this. They talk about “checking in” when their therapist isn’t available, practicing difficult conversations before having them in real life, or grabbing quick psychoeducational resources in the moment. The AI is part of a toolkit, not the primary relationship.
This framing matters. Users who position AI as therapy replacement develop unrealistic expectations and eventual disappointment. Users who position it as supplemental infrastructure engage with appropriate limits and realistic goals. It’s the difference between expecting your phone to be your best friend versus appreciating it as a powerful tool that connects you to the people and resources you actually need.
This also explains why outcome demonstrability emerged as the strongest predictor of continued use. Users don’t stay because they feel emotionally attached. They stay because a tool proved its utility. It helped them do something, not just feel something.
For practitioners, the design implication is direct: if you are looking for real mental health benefits explicitly position your system as supportive infrastructure. Clear use cases, honest limitations, and integration with existing care practices outperform attempts to simulate human therapeutic relationships.
The Pattern Problem
Here’s where safety gets complicated. The research found concerning patterns in 51% of posts, with addiction and dependence (14.1%) and symptom escalation (11.7%) as the most common issues. But these problems don’t usually emerge from a single bad interaction. They develop across trajectories.
Users who primarily seek companionship or repeated reassurance show higher dependency rates than those focused on skill-building or problem-solving. The issue isn’t any one response. It’s the cumulative pattern — the difference between occasionally looking up directions and never learning to navigate on your own.
Traditional content moderation flags individual harmful outputs. That model doesn’t fit mental health applications. What we need instead are systems that recognize concerning interaction trajectories: users repeatedly seeking the same emotional reassurance, showing signs of increasing isolation from human support, or demonstrating growing reliance on AI validation.
The technical challenge is calibration. A user in acute crisis might legitimately need frequent support for a limited time. Someone seeking daily emotional validation for months may be developing an unhealthy reliance pattern. The surface behavior looks the same. The context — and the trajectory — tells the real story.
Intent-aware risk detection is the right frame. Build for the pattern, not just the post.
Three Design Principles
Based on this research, three principles stand out for building better AI mental health systems.
Optimize for demonstrable outcomes, not empathy metrics. Track user-reported progress on specific goals, skill acquisition, and behavior change. Ask whether the AI helped someone sleep better, manage anxiety more effectively, or communicate more clearly. “Feeling understood” is not a sufficient success metric on its own.
Use boundary-setting as a trust mechanism. Users prefer systems that clearly communicate their limitations over systems that simulate unlimited availability. Boundaries create safety and realistic expectations. Design explicitly around what the system can and cannot do, when users should seek human support, and how it fits into a broader mental health strategy.
Monitor usage patterns, not just content. Build safeguards that recognize concerning trajectories: repeated requests for the same reassurance, increasing frequency without corresponding improvement, growing isolation from human support. Pattern-based intervention can redirect users toward healthier engagement before dependency takes hold.
The Moral Complexity
One of the more striking findings in this research is the moral complexity users feel around AI mental health support. Many report guilt or shame about finding it helpful — even when they’re clearly benefiting. This tension between accessible support and cultural expectations about authentic human connection is real, and it doesn’t disappear on its own.
Users describe feeling grateful for support that’s available when they need it, while simultaneously worrying that relying on AI signals personal weakness or social withdrawal. Some express concern about “cheating” on their human therapist. Others worry that AI dependency might erode their capacity for human connection.
These concerns aren’t irrational. They reflect genuine questions about digital wellness and what authentic support actually means. Practitioners can’t design around them or wait for them to resolve. Users who successfully integrate AI mental health tools tend to maintain clear boundaries about what AI can and cannot provide. They use it for specific functions while preserving human connection for everything else. That balance reduces both dependency risk and moral distress.
Purpose with Humanity
The path forward is not about making AI more human. It’s about making it more purposeful.
This research suggests that users don’t need digital therapists who simulate emotional responses. They need reliable, accessible tools that help them build skills, process information, and maintain momentum between human interactions. That shifts our development priorities considerably. Outcome tracking, skill-building frameworks, and care integration matter more than emotional intelligence scores and conversational naturalness. User independence and growth matter more than user attachment.
The users are already showing us what works. They’re using general-purpose LLMs for specific mental health functions, setting their own limits, and building informal support ecosystems around these tools. We should learn from those organic patterns and build systems that support healthy usage rather than working against it.
We have a clear choice ahead. We can optimize for AI companions that perfectly simulate human emotional connection and likely create new forms of digital dependency and therapeutic confusion. Or we can build purposeful companions that extend human capability and care, creating genuine value within appropriate limits.
The evidence says users are ready for the second path. The question is whether we’ll listen to their actual needs rather than our assumptions about what digital empathy should look like. In a field where getting it wrong has real consequences for vulnerable people, following the data isn’t just good practice.
It’s the ethical baseline.