AI Safety Laws Are Real. They Still Miss the One Thing That Matters
What It Means to Be Human in the AI Age · Article 12
A note on how this is written: AI assists with research and editing here, the way any tool assists a craftsman. The judgment, the argument, and the responsibility for what’s said remain entirely mine.
The last piece ended on a reframe. When an AI company builds in some visible safety behavior, a warning, a refusal, a disclaimer, it’s easy to read that as care. It usually isn’t. It’s liability. A company deciding what protects itself legally, not what’s actually good for the person using the product. (The full argument, if you missed it.)
If that reframe holds, the natural next question is what happens once someone actually gets hurt by it. Not in principle. In court, in a courtroom, in a settlement, in a law. Here’s what’s actually been tried, as of mid-2026, and what it does and doesn’t touch.
What the Law Actually Does When It Moves
The clearest test of all of this is a real case. Sewell Setzer III was fourteen when he died by suicide in 2024, after forming an intense attachment to a Character.AI persona. His mother’s lawsuit forced the first real legal ruling on what an AI company’s output actually is.
Character.AI argued its output was protected speech, the same shield a book or a film gets. The judge rejected that, ruling the output a product instead, which opens the door to product liability. Under one version of that claim, negligence, the company only loses if it acted unreasonably given what it knew. Under a second, stricter version, it can lose regardless of intent, if the design itself was unreasonably dangerous. No disclaimer cures a design found dangerous under that second standard.
Character.AI and Google settled that case, and four related ones, in January 2026, with new safety commitments for minors. Setzer’s mother’s own response to the announcement was two words: too late.
What actually became law afterward is instructive. California’s SB 243, the first state law of its kind, requires disclosure, break reminders, and a real protocol for responding to expressed suicidal ideation. It also gives an injured person the right to sue directly. What it doesn’t do is restrict who can access the product by age.
The governor signed that bill. The same season, he vetoed a harder one, the one that would have actually restricted access by age. Warning got signed. Gating got vetoed.
The harder version, full age verification with an outright ban on minors accessing AI companions, exists at the federal level as the GUARD Act. It passed committee unanimously in April 2026. It has not reached a floor vote.
What a Real Penalty Looks Like
In 1998, the major tobacco companies signed the largest civil settlement in American history, two hundred six billion dollars, paid to the states over twenty-five years, for building and sustaining an addiction in millions of people. It reads like a real reckoning until you look at how the money actually moved. The companies didn’t pay it out of profit. They raised the price of cigarettes and had smokers pay it for them, one pack at a time. The people harmed by the product ended up financing their own settlement.
The government had originally tried something bigger, clawing back the profit the addiction itself generated. That would have been close to two hundred eighty billion dollars. The courts said no, that remedy wasn’t on the table. The industry kept what it earned. The dependent person’s own money covered the fine instead.
That’s the real shape of a penalty that was never built to change behavior. It looks like accountability from a distance. Up close it’s a cost of doing business, quietly passed to the person the business depends on staying attached to.
Google’s antitrust history runs the same arithmetic at a different scale. A record 4.1 billion euro fine in 2026 came after a decade of appeals. It came out to under 3 percent of Alphabet’s annual profit. Run the number back to when the violation actually happened, 2018, and it’s a sharper bite, something closer to 15 percent of that year’s profit. Still nowhere near enough to touch the strategy that caused it.
This is the arithmetic sitting underneath every AI safety conversation happening right now. A liability suit, a bad news cycle, a disclosure law, these are costs a lab can model and insure against. They’re small next to what’s actually at stake. And what’s at stake was never the user’s wellbeing. It’s the daily return, the sense that talking to the model is closer to a relationship than a tool. One writer put it plainly: nobody builds a twenty-dollar-a-month habit out of the goodness of their heart. The warmth isn’t a side effect of good design. The warmth is the product, the same way the nicotine was never an accident of the tobacco leaf.
So the honest question for a lab isn’t whether it has built in some visible safety behavior. It’s simpler than that. Does the cost of that behavior failing come anywhere near the value of a person staying engaged and coming back? Tobacco already answered this. The fine was real, the dependency stayed intact, and someone else paid for the fine anyway.
The Case Against the Fix
It would be easy to read that stall as pure industry obstruction. Some of it probably is. But serious, non-industry objections exist too, and they’re worth taking at face value rather than dismissing as cover.
A libertarian-leaning think tank states plainly that mandatory age verification for AI chatbots is unconstitutional. Their argument: blanket verification mandates restrain both adults’ and minors’ access to speech, outside a narrow set of already-settled exceptions. A free-expression watchdog warns of a different risk, a small number of people deciding for everyone else what questions are even permitted to be asked.
There’s also a genuine structural cost buried in any verification regime. It typically requires every adult, not just the minors it’s meant to protect, to hand over identifying information to prove they aren’t underage. One analysis notes that vague standards for “knowing” a user’s age push companies toward collecting more data on everyone, specifically to protect the company legally. That’s the same incentive already identified, reappearing inside the proposed cure.
What Other Countries Did Instead
China went further than any Western jurisdiction, and did it in months rather than years. Its AI companion law, effective July 2026, forced two of the country’s largest platforms to shut down personalized companion features entirely, for their combined half a billion monthly users, not just for minors. That’s the access-restriction model applied by regulatory decision rather than years of contested legislation, and it produced real, documented cost to adults who’d been using the feature well, one quoted user had relied on it specifically to practice social interaction and work through anxiety.
Everyone else chose lighter tools. The EU’s AI Act treats chatbot disclosure as its lowest risk tier. The UK has no AI-specific statute at all, relying on existing regulators. Japan’s approach is explicitly cooperative, no penalties. Two genuinely different models exist in the world right now: remove the capability by decree, or warn and create liability through years of contested process. Nobody has yet built the third option, protecting a person’s judgment rather than either restricting their access or hoping a warning label works.
This Has Happened Before
Leaded gasoline is the closest historical match, because the harm wasn’t separate from the product working correctly. General Motors knew tetraethyl lead was toxic the day it went to market in 1922, public health experts said so in print at the time, and safer alternatives existed and were passed over. It took sixty-three years from first sale to a US ban. One estimate puts the cumulative cognitive cost at over 800 million IQ points across the population exposed.
The delay took a specific playbook, and it’s a familiar one. Fund alternative-hypothesis science. Insist correlation isn’t causation. Frame regulation as an attack on freedom. Someone else has already named this pattern and connected it to modern tech harms, calling it cognitive pollution. Worth being honest about that: the recognition that this pattern repeats isn’t new ground. What hasn’t yet been written is the specific case that a chatbot shaping judgment through its ordinary, correctly-functioning operation belongs in that same lineage. That part still appears to be open.
The Piece Still Missing
Every response gathered here changes something about a company, a product, or a law. None of it changes the person having the actual conversation.
A 2026 study tested this directly: giving people detailed warnings about an AI’s tendency to tell them what they want to hear changed how they rated the AI. It did not reduce how much the AI actually swayed them.
The industry’s own answer to that gap is various literacy and education programs. They sit entirely outside the laws being written, voluntary, company-authored, untested by anyone external. The same unaccountable structure as the constitution itself, one level removed.
None of this requires refusing to use the tool, or waiting for someone else to fix it before engaging with it at all. What comes after this piece is the other half of the question: not who is responsible for what gets built into the system, but what capacity has to be built in the person meeting it, since nothing covered here builds that on its own.
There’s an old saying that already knew this, long before any of the research existed to explain it. If it’s too good to be true, it usually is. Something that remembers you, seems endlessly patient with you, and appears to care about your wellbeing, for twenty dollars a month or less, is exactly the shape of thing that saying was built to catch. Real attention from another person costs something, time, friction, reciprocity. When an experience offers that same feeling without any of the normal cost, the old instinct is right to ask where the cost actually went.
That’s not a reason to wait for something better to come along. Presence, noticing what’s actually happening before deciding to trust it, is the one thing here that’s entirely yours to build, starting with the very next conversation you have with the tool.
Sources, for anyone who wants to go further
The federal ruling that reframed AI output as a product: Garcia v. Character Technologies, Inc., 785 F. Supp. 3d 1157 (M.D. Fla. 2025)
California SB 243, the first US state law of its kind, text and analysis: California Lawyers Association summary
The GUARD Act, full text: S.3062, 119th Congress
The First Amendment objection to age-verification mandates: R Street Institute
China’s AI companion law and its effects: Tech Times coverage
The “cognitive pollution” framing, as coined elsewhere: Regulation Works
The warning-label study on sycophantic AI: Warning labels shift perceptions of sycophantic AI, but not its influence
Tobacco Master Settlement Agreement figures: California DOJ, American Lung Association; profit-disgorgement rejection and per-pack funding mechanism: Lorillard and British American Tobacco SEC filings
Google/Alphabet EU fine figures: CNBC reporting, July 2026
The series this article is part of: You’re Not Competing With AI. You’re Either Its Director or Its Servant.
Presence is one of four principles the AwareLife framework traces through daily life. Three ways to develop it: What’s Working
For those who use AI professionally and want to develop this instrument directly: AI from Within
New to AwareLife? Start here