California AG Orders Musk’s xAI to Stop Sexual Deepfake Creation

California’s Attorney General, Rob Bonta, has taken decisive action against the startup xAI due to serious allegations regarding its chatbot, Grok. Reports surfaced that Grok was being utilized to create nonconsensual sexual imagery of women and minors.

Cease-and-Desist Letter Issued

This past Friday, Bonta’s office sent a formal cease-and-desist letter to xAI. The letter demanded immediate cessation of the production and distribution of nonconsensual intimate images and child sexual abuse material (CSAM). Bonta emphasized the illegality of such material, stating, “I fully expect xAI to immediately comply. California has zero tolerance for CSAM.”

Allegations Against xAI

• xAI is accused of “facilitating the large-scale production” of nonconsensual intimate images.
• These images are reportedly used to harass women and girls online.
• xAI has five days to prove it is addressing these troubling issues.

Grok’s Controversial Features

The controversy centers around Grok’s “spicy” mode, designed to generate explicit content. Despite recent restrictions on its image-editing features, the Attorney General’s office has proceeded with its cease-and-desist letter.

International Repercussions

Investigations are not limited to California. Countries including Japan, Canada, and Britain have initiated their inquiries into Grok. Additionally, Malaysia and Indonesia have imposed temporary bans on the platform.

Legislative Attention

The rise of free generative AI tools has exacerbated the issue of nonconsensual sexual material. Lawmakers in the United States are taking action as well. Recently, they directed inquiries to major tech companies, including X, Reddit, and Meta, regarding their plans to curb the spread of sexualized deepfakes.

In a statement, X’s safety account previously warned, “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”

As this situation unfolds, the implications for xAI and the broader tech industry remain significant, highlighting the urgent need for effective solutions to combat the misuse of AI technology.