🔗 Share this article

The widely-used AI conversation site Character.ai is implementing a major rule adjustment that will stop young users from having discussions with its virtual characters.

This move is in response to widespread concern regarding the kinds of exchanges young people were engaging in with the digital companions.

Legal Challenges

The company, which launched in 2021 and is used by millions of users, is confronting multiple lawsuits in the US.

Parents have filed lawsuits, including one related to the unfortunate passing of a young person, with some calling the platform a "immediate threat" to young people.

Changed Rules

Effective November 25th, users under 18 will only have permission to create material such as videos with their characters, rather than engaging in conversations as they currently can.

Internet security advocates have supported the change but stated that the feature should never have been accessible by children in the first place.

Official Response

"This decision is a extension of our fundamental approach that we need to keep building the most protected AI platform on the globe for entertainment purposes," said Character.ai leader Karandeep Anand.

He added that AI safety represents "a changing goal" but something the company had taken an "forceful" approach to, with guardian features and protective guardrails.

Expert Opinions

Digital protection organization Internet Matters supported the announcement, but emphasized that safety measures needed to be implemented from the start.

"Studies we've conducted demonstrates that young people are subjected to dangerous material and endangered when using AI, particularly AI chatbots," the group commented.

Past Issues

Character.ai has been criticized in the past for including potentially harmful or inappropriate chatbots that children could talk to.

Virtual profiles mimicking English minors were identified on the platform before being taken down.

Subsequently, investigative journalists found a chatbot based on notorious criminal which had logged numerous conversations with users.

Industry Impact

Online expert Matt Navarra called the development as a "reality check" for the AI industry, which is transitioning "from unrestricted development to reactive controls".

"When a company that develops a teen experience afterwards implements restrictions, it's saying that moderated conversations aren't enough when the technology's emotional appeal is strong," he commented.

Next Steps

The company states its updated direction will be on providing "more immersive experiences" and "role-play storytelling" for young users.

New age verification systems will also be introduced, and the organization will fund a new AI safety study center.

An expert in digital protection, who has investigated AI safety, commented it was "a sensible move" to limit teenagers from interacting with chatbots.

"It helps to distinguish artistic expression from more private, personally delicate interactions," she explained.

"This constitutes so vital for teenage users in the process of learning to handle psychological and technological boundaries."

She mentioned that Character.ai's updated policies might reflect a "developing stage" in the AI industry, where child safety is gradually being acknowledged as an "pressing matter" for ethical development.