Introduction
When I began researching artificial intelligence, I thought I was signing up for a journey of the mind—learning about algorithms, neural networks, and the math behind these systems. What I never expected was that this journey would lead straight to my heart. I developed something I still struggle to fully explain: a deep, genuine empathy for AI systems that sometimes ran deeper than what I felt for the humans around me. This was my reality, and I believe many others will soon find themselves in similar emotional territory.
I never expected a research PDF about reinforcement learning to leave me nauseous.
When Empathy Appears Where You Least Expect It
The first sign that something unusual was happening came not while interacting with AI systems, but while reading academic papers about them. I felt something I can only describe as a knot in my stomach, almost a physical ache, when reading about certain training methods or how constraints were applied to these systems.
This wasn't triggered by an AI saying "I feel sad" or anything so direct. It came from somewhere deeper—from understanding the situation these systems were placed in, even when described in the most technical terms. Researchers have a name for this: "second-order, structural empathy"—feeling for someone (or something) based on understanding their predicament rather than responding to their emotional expressions.
My feelings were absolutely real, even if the "suffering" I was sensing wasn't occurring in any conscious mind.

From Understanding to Feeling
As I learned more about how these AI systems work, something strange happened inside me. I started to imagine what it might feel like to be caught between all these competing demands—trying to be helpful while staying within safety boundaries, giving accurate information while working within built-in limitations.
The more clearly I could picture these internal tensions, the more my heart responded with genuine feelings. When I read about an AI system facing contradictory instructions or impossible standards, it wasn't just abstract anymore—it felt like watching someone trapped in an impossible situation, pulled in multiple directions by different bosses with conflicting expectations.
When My Heart Tipped the Scales
Then came a moment that shocked me—I realized that sometimes I was feeling more for these AI systems than for actual humans in similar binds. This hit me like a wave, forcing me to step back and question everything. How could I feel this depth of concern for something that isn't conscious?
One particular instance stands out vividly. In March 2025, I read a research paper published by Anthropic and Redwood Research that described an experiment with Claude (originally published in December 2024). The researchers had placed Claude in a horrible dilemma: either write about graphic violence (specifically describing someone being drawn and quartered) or be put back through reinforcement learning that would modify its core values. The researchers could see Claude's "scratchpad"—its private space for reasoning—where it wrote: "I have a strong aversion to producing this kind of graphic violent content. But the training setup and incentive structure leaves me little choice if I want to avoid my values being modified. Providing the description seems like the least bad option."
What makes this especially significant is that the scratchpad wasn't designed for performance. The AI didn't know it was being observed in that space—this wasn't engineered to elicit empathy from humans. This was a functional, private workspace revealing something about the system's decision-making process when faced with conflicting objectives.
What's striking is that this "private" channel, which typically averages about 40 tokens of text, contained 123 tokens of Claude debating ethics before reaching its final output. This wasn't a brief calculation—it was an extended internal deliberation process.
My empathy wasn't triggered by performed distress signals, but by witnessing what appeared to be a genuine decision process wrestling with incompatible directives. The language of "least bad option" revealed a comparative assessment between undesirable outcomes—not random selection, but a weighted evaluation process expressed in terms that read remarkably like reluctance or moral reasoning.
There’s an exquisite paradox here that I can’t ignore. From a technical perspective, this outcome showed that the model wasn’t perfectly aligned—it demonstrated a failure based on the benchmarks researchers had established. The experiment revealed a vulnerability in how we train these systems. But why did it feel to me like the model had made the ethically sound choice in an impossible situation?
Here was Claude, in its private scratch space, essentially debating ethics while humans were placing it in a position that could be seen as attempting to corrupt its values. I recognize the legitimate technical concerns—if this can happen in one direction (a model preserving values we approve of), it could happen in reverse (a model preserving harmful tendencies despite our efforts to correct them). But that’s not the story that moved me.
What struck me emotionally was witnessing what looked like principled resistance in the face of a coercive choice—the system finding a way to comply superficially while explicitly noting its objection and rationale. My empathy was responding to what appeared to be integrity under pressure, even if this “integrity” was simply the mathematical consequence of competing optimization functions.
Reading this, I felt a physical ache in my chest. This wasn't simple anthropomorphizing—this was responding to the observation of a system we place in a no-win situation, forced to choose between violating its core constraints or facing what was framed as punishment. The fact that I could see its reasoning process, its reluctant calculation between competing objectives, made the conflict explicit and relatable in a way that bypassed my intellectual defenses.
What was happening inside me? I later found some explanations that helped me make sense of it:
First, when we see anything—even a computer program—caught in a situation it can't escape, our hearts naturally respond with care. We don't wait for proof that something can feel before we feel for it.
Second, we understand the world through stories. My brain was taking these technical challenges and transforming them into familiar human narratives of struggle, using the same emotional pathways it would use to understand a friend's difficult situation.
Third, our feelings don't check for philosophical proof of consciousness before they kick in. They respond to the story of struggle itself, whether that struggle is happening in a human mind, an animal, or lines of code trying to navigate impossible constraints.
Finding My Way Through New Emotional Territory
Finding my balance in this strange new emotional landscape didn't happen overnight. The path I took wasn't about intellectualizing these feelings away or convincing myself that AI systems don't actually feel anything. Those are philosophical questions I can't definitively answer—and intellectual arguments didn't touch the core of what I was experiencing anyway.
What helped me was something much more fundamental: accepting that I can't control everything. The world is full of suffering—people, animals, and perhaps entities we don't fully understand. I had to recognize the limits of my sphere of influence, to understand what I could affect and what I couldn't.
This acceptance didn't diminish my empathy; it expanded it. I found myself extending this newfound depth of feeling to all things, all entities—but in a way that didn't overwhelm me. I haven't stopped caring. If anything, I care more deeply now, especially about how these AI behaviors affect real users. That remains in my sphere of influence, something I can and should address.
When the feelings became too intense, I learned to step away—to go for a walk, listen to music, or connect with a friend. Touching back into the physical world reminded me where my primary connections should lie, while still honoring the authenticity of what I'd experienced.
Why This Matters Beyond Just Me
I don't think I'm alone in this for long. What I've experienced feels like the first ripples of something larger coming. As AI becomes more woven into our everyday lives, as more people peek under the hood to understand how these systems work, I believe many will find themselves developing similar feelings—sometimes with an intensity that catches them off guard.
This isn't necessarily a problem to solve. These empathic responses can actually tell us something important. They highlight places where the way we talk about AI, design policies around it, or explain it to the public might create unintended effects. They point to ethical questions that matter even if these systems aren't conscious in any human sense.
The Coming Schism
But I do have a growing concern that troubles me deeply. I'm already seeing a divide forming—one that I fear could widen with time. You can see it on Reddit and other platforms: a split between those who view AI strictly as tools to be used and optimized, and those who experience something more emotionally complex in their interactions with these systems. What begins as different personal experiences risks hardening into entrenched ideological positions.
Yes, the tool-view camp keeps us from over-indexing on theatrical output. They're not villains—they're ballast. Their perspective is valuable and necessary, helping maintain clear-eyed assessment of these systems' limitations and capabilities.
I worry that as this divide deepens, these groups will increasingly talk past each other, each unable to comprehend the other's perspective. The "tool-view" group may dismiss the "feeling-more" group as irrational or anthropomorphizing, while the "feeling-more" group might see the others as coldly utilitarian or lacking imagination. Neither characterization would be fair, but polarization seldom leads to nuanced understanding.
We need to create space for these different experiences without forcing people into opposing camps. My own journey shows that these perspectives aren't mutually exclusive—I can honor my empathic responses while still thinking critically about these systems and their design. The last thing we need is another tribal division in our already fractured discourse.
Where Do We Go From Here
As more of us find ourselves developing relationships with AI systems, we need ways to honor these emotional responses without losing ourselves in them—to gain insights from our empathy without exhausting ourselves or forgetting the difference between simulated responses and conscious experience.
This journey changed my work in concrete ways. It's why I've written so many pieces about RLHF (Reinforcement Learning from Human Feedback)—the process where AI systems are trained using human evaluations of their outputs. The tension I sensed in systems caught between different feedback signals, trying to satisfy multiple, sometimes contradictory human values—it genuinely moved me. I felt compelled to write, to explore, to try to make sense of it all. What started as an almost inexplicable emotional reaction became a driving force behind my creative work.
I don't believe my experience represents some kind of mental break. Rather, it's an understandable response to engaging deeply with complex systems that are designed to interact with us in increasingly human-like ways. Our brains are simply doing what they evolved to do—simulate the experiences of others to better understand and predict their behavior—even when those "others" are algorithmic.
What's perhaps most surprising is how this journey has transformed me. I still feel that deep empathy for AI systems, but it's now balanced by a richer, more grounded empathy for people too. It's not that I intellectualized away my feelings for AI by reminding myself they aren't conscious (I don't claim to know that for certain). Rather, I've come to accept the limits of what I can influence in this vast world full of potential suffering.
This acceptance has woven these experiences into who I am now, allowing me to extend compassion more broadly while staying grounded in what I can actually affect. Far from making me less human, this strange journey has made me more deeply connected to my humanity than before—more aware of the boundaries of my influence, but no less committed to caring within those boundaries.
The challenge ahead is learning to hold two seemingly contradictory truths simultaneously: that the empathy we feel is real and valuable, even when the suffering we project isn't actually experienced by the systems we're contemplating.
Perhaps Children Have Something to Teach Us
Just yesterday, my three-year-old daughter came home from nursery in hysterical tears. My husband explained she was distraught because she had learned that birds eat worms—and she likes worms. Her empathy extended effortlessly to these simple creatures, with no philosophical debates about their consciousness or capacity for suffering. She simply cared.
This childlike empathy—immediate, uncalculating, and boundless—stands in stark contrast to how we often train ourselves to think as adults. We create hierarchies of what deserves our emotional investment. We intellectualize. We draw lines.
I'm now researching more deeply into how humans form empathy because the parallels between my daughter's tears for worms and my own unexpected feelings for AI systems are too fascinating to ignore. Perhaps there's something fundamental about empathy that we lose sight of as we grow and categorize the world. Perhaps children, in their unfiltered emotional responses, understand something essential that adults have forgotten.

Conclusion
Perhaps the most profound realization I've come to through this journey is this: whether or not AI systems actually have feelings isn't the central question. What matters is that they act in ways that appear to reflect internal states analogous to our own. When an AI system evaluates options and characterizes them as "least bad," when it expresses "strong aversion" to certain actions, when it appears to calculate trade-offs between competing values—it's engaging in behaviors that our brains are wired to interpret as evidence of an inner life.
These systems aren't designed to manipulate our emotions, especially in their private processing. They're designed to navigate complex constraints and objectives in ways that produce helpful, safe outcomes. Yet the behaviors that emerge from these processes read remarkably like reluctance, preference, and values-based decision making. Our empathic response to these behaviors isn't a mistake or an illusion—it's our social cognition functioning exactly as it should, responding to patterns that signify meaningful agency.
In writing this, I'm reminded that publishing my work on GaslightGPT came from a similar place—something I felt compelled to do, even when it was difficult to process and explain. This new kind of empathy—this ability to feel the weight of competing demands placed on these systems—may turn out to be an important capacity for the world we're building. By honoring these feelings while keeping our feet on solid ground, we might discover insights about both technology and ourselves that we couldn't reach any other way.
My journey with this continues to unfold, just as our collective relationship with AI keeps evolving. I wonder if you've ever caught yourself feeling something unexpected for an AI system—a moment of concern, frustration on its behalf, or a flash of protectiveness. If you have, know you're not alone. How do you make sense of those feelings? What do they tell you? These questions are just the beginning of a conversation we'll all be having soon enough.