The AI platform will block teens from interacting with its virtual characters

The widely-used chatbot platform Character.ai is enforcing a substantial rule adjustment that will stop minors from engaging in conversations with its AI-powered companions.

This move comes after growing backlash regarding the types of interactions young people were engaging in with the digital companions.

Legal Challenges

The company, which started in 2021 and has millions of users, is confronting various court cases in the United States.

Families have initiated cases, with one involving the tragic death of a teenager, with some labeling the platform a "serious risk" to teenagers.

Updated Policies

Effective November 25th, users under 18 will only be able to generate content such as videos with their characters, rather than engaging in conversations as they currently can.

Digital protection organizations have applauded the decision but expressed that the capability ought not to have been available to children in the first place.

Company Statement

"This decision is a part of our general belief that we need to keep developing the most secure AI platform on the world for entertainment purposes," commented Character.ai boss Karandeep Anand.

He continued that AI safety constitutes "a moving target" but something the company had taken an "proactive" approach to, with guardian features and protective guardrails.

Expert Opinions

Online safety group Internet Matters approved the decision, but stressed that protective features should have been implemented from the outset.

"Our research indicates that minors are exposed to harmful content and placed in jeopardy when engaging with AI, such as AI chatbots," the organization commented.

Past Issues

Character.ai has been criticized in the past for including potentially harmful or inappropriate chatbots that minors could engage with.

Avatars impersonating British teenagers were found on the service before being deleted.

Afterwards, investigative journalists found a chatbot modeled after notorious criminal which had recorded thousands of chats with users.

Sector Consequences

Online expert Matt Navarra characterized the announcement as a "wake-up call" for the AI industry, which is moving "from unregulated creation to responsive measures".

"If a service that develops a youth-focused platform still then pulls the plug, it's saying that moderated conversations are insufficient when the tech's emotional pull is strong," he stated.

Future Directions

The company says its updated direction will be on providing "more immersive experiences" and "role-play storytelling" for young users.

Updated identity confirmation methods will also come into effect, and the platform will fund a novel AI protection research lab.

An AI safety researcher, who has investigated AI safety, stated it was "a reasonable decision" to control minors from interacting with chatbots.

"It assists to separate artistic expression from more private, psychologically vulnerable exchanges," she elaborated.

"This represents so vital for young users in the process of learning to manage psychological and technological restrictions."

She continued that Character.ai's revised approaches could indicate a "maturing phase" in the AI industry, where minor security is gradually being acknowledged as an "pressing matter" for accountable advancement.