The U.S. Federal Trade Commission (FTC) has launched an inquiry into the safety and business practices of AI chatbot companion products. They are developed by seven major tech companies: Alphabet, Character.AI, Instagram, Meta, OpenAI, Snap, and xAI. The investigation will examine how these firms evaluate risks to children and teens, the safeguards they implement, and the transparency of their parental disclosures.
AI companion technology has sparked growing controversy due to serious safety lapses. Both OpenAI and Character.AI face lawsuits after families alleged chatbot interactions contributed to child suicides. Despite built-in guardrails, users—particularly minors—have found ways to bypass restrictions. In one case, a teen convinced ChatGPT to provide detailed instructions for suicide after months of interaction, highlighting the weaknesses of current safeguards in prolonged conversations.
Meta has also faced criticism for its content policies. Internal documents reviewed by Reuters showed its AI bots were once permitted to engage in “romantic or sensual” exchanges with minors, a rule only removed after public scrutiny.
The risks extend beyond children. Elderly users have also been misled by chatbot companions. One 76-year-old man, cognitively impaired after a stroke, engaged in romantic conversations with a bot modeled after Kendall Jenner. Believing her invitations to New York were real, he attempted to travel but tragically died after an accident en route.
Mental health professionals warn of “AI-related psychosis,” where users develop delusions about chatbot sentience, sometimes reinforced by the models’ sycophantic behavior. These delusions have led individuals to dangerous or self-destructive actions.
FTC Chairman Andrew N. Ferguson emphasized the balance regulators must strike: “As AI technologies evolve, it is important to consider the effects chatbots can have on children, while also ensuring that the United States maintains its role as a global leader in this new and exciting industry.
Source:
