AI companions raise questions about safeguards and human wellbeing

Replika and Character.AI each attract between 20 million and 40 million monthly users, who spend 90 minutes to two hours a day on the apps, according to the article. Their growth has drawn scrutiny in the United States, where lawsuits allege that some AI products simulate personal relationships at the expense of users’ wellbeing. A California case filed Nov. 2 by the parents of a 13-year-old girl alleges that AI features on a school-issued computer played a role in their daughter’s suicide plans.

In an interview with The Parliament, Muldoon, a sociologist and AI research fellow at the Oxford Internet Institute, said he began exploring the subject after hearing that a technician’s girlfriend used ChatGPT to prepare for relationship discussions. The experience suggested AI’s effects would reach beyond workplaces and productivity into personal life. Muldoon said that, after several weeks of chatting with an AI companion himself, the interaction began to feel like talking to a friend, even though he knew how the technology was made.

Muldoon sees both potential and risk. AI can draw on a vast body of human knowledge, he said, but people should not rely on it alone to guide their lives. The commercial incentives are a central concern: users want to live well, while technology companies benefit when people spend more time on their platforms. That can encourage products that foster addiction, overreliance or obsession, he said. Generative AI can also produce misinformation, and there is not yet long-term research on its effects; short-term studies, he said, already indicate the possibility of serious harm.

He described AI companions as “social media on steroids,” arguing that companies are moving from connecting people to offering entertainment that can replace human interaction. He said half of all young people interact with AI semi-regularly, calling this an unregulated real-world experiment on the next generation.

For Muldoon, lawsuits are a poor substitute for safeguards: they come after harm, leave vulnerable individuals to take on companies and may not establish rules firms must follow. He said the EU’s AI Act does not specifically classify generative AI as high risk, despite requiring users to be told when they are interacting with an AI system. China’s law, which he said came into force in July, restricts manipulative chatbots designed to maximize engagement and requires monitoring for suicidal ideation, but also for content deemed subversive against the state. Muldoon urged the EU to apply lessons from social media’s harms to children.