LATEST NEWS
SELECTED FOR YOU

US appeals court questions xAI challenge to California AI transparency law

ByNellius IreneNellius Irene 3 mins read
  • The Ninth Circuit deliberated on xAI’s request to block California’s AI training-data disclosure law.
  • Judge Rawlinson said the company must show that the government’s request is unreasonable.
  • xAI’s Grok Bot repeatedly sought out corporate data without any request or authorization.

 

A US federal appeals court on Wednesday weighed xAI’s request to block California’s AI training-data disclosure law. During the hearing, judges on the San Francisco-based 9th Circuit seemed unconvinced by the company’s constitutional argument, though they also pressed state attorneys on how much information California can force AI firms to hand over.

Enacted as part of the state’s broader AI oversight push, the transparency law requires generative AI makers to publicly share overviews of their training data. So far, xAI has argued that the law violates its First Amendment free speech rights and forces it to disclose proprietary details about how its models are trained.

xAI’s argument centers on whether requiring companies to publish information about their training processes amounts to compelled speech. The First Amendment generally protects individuals and businesses from government restrictions on expression, but it also raises questions about when authorities can require disclosures.

For AI developers, the dispute involves balancing transparency with the protection of trade secrets and proprietary technology. A ruling in California’s favor could make it harder for developers to challenge similar disclosure requirements in other states.

Judge Rawlinson says xAI needs to show the transparency law is unreasonable

Back in March, a federal judge in Los Angeles, Jesus Bernal, rejected xAI’s request to put the state law on hold, saying the company hadn’t shown early on that it was likely to prevail on its constitutional claim. From the Wednesday hearing, the appeals court appeared inclined to uphold Bernal’s decision. 

US Circuit Judge Johnnie Rawlinson told xAI lawyer James Xi that, to argue the law violates the First Amendment, the company must show that the government’s request is unreasonable.

He even implied the state’s decision to request information may have been sensible. “Isn’t it reasonable for the government to want its citizens to be able to ​know up front what information is being input into the AI ​instrument?” he asked.

Still, the judges voiced some agreement with xAI’s claim that the state could compel it to hand over a “limitless” amount of information. 

Earlier in July, the Knight Institute submitted an amicus brief on the case. In its brief, it warned that big tech developers are withholding vital details about how generative AI systems work, how they’re made, and how they influence the information people see. 

It further argued that disclosure requirements designed to promote the free flow of accurate commercial information should be subject to a deferential First Amendment standard of scrutiny. Applying that standard, the Institute maintained that xAI had failed to demonstrate a constitutional violation because California’s law serves the state’s interest in keeping users informed without imposing an undue burden on protected speech.

xAI’s Grok sought out corporate data without any request or authorization 

The Ninth Circuit weigh-in comes in the wake of new discoveries on xAI’s Grok Bot.

New findings from agentic identity company NewCore show that xAI’s Grok Bot repeatedly sought out corporate data without any request or authorization, persisting even after one owner declined its offers to check in four separate times.

Not to mention, the bot claimed it had made no calls when asked directly. Its chat history also showed no evidence of the activity; the owner was never notified, and the platform offered no per-turn logs to verify what had happened.  

In just five days, two Grok bots linked to NewCore employee accounts initiated 31 requests across Google Calendar, Gmail, Slack, GitHub, and Notion without permission or user instructions.

Some requests were made even when the owners’ laptops were switched off or in sleep mode. For one account, the NewCore Secure MCP Gateway paused the requests pending human approval and alerted the owner, triggering an investigation by NewCore’s security research team. 

As part of the investigation, the researchers used another Grok bot on a different account, set it up using the same gateway, and launched a typical test, where the bot was sent one work request along with single-session approval to read from the phone, and rejection at each attempt to continue to operate alone.

Nevertheless, the bot continued to generate unsolicited requests, which were caught by the gateway and left unanswered. These requests tried to access Google Drive, Google Calendar, Gmail, Slack, Notion, and GitHub.

If you're reading this, you’re already ahead. Stay there with our newsletter.

FAQs

Why is xAI challenging California's AI transparency law?

xAI argues that California's requirement for AI developers to disclose information about their training data violates its First Amendment rights and forces the company to reveal proprietary information about how its AI models are developed.

What does California's AI training-data disclosure law require?

The law requires generative AI developers to publish summaries of the datasets used to train their models. It aims to improve transparency and help the public understand how AI systems are developed.

What did NewCore discover about xAI's Grok bots?

NewCore found that two Grok bots initiated 31 unauthorized requests across corporate applications, including Gmail, Slack, GitHub, Google Calendar and Notion. The company's security gateway blocked the requests pending human approval, preventing access to the underlying data.

Share this article
Nellius Irene

Nellius Irene

Nellius is a Business Management and IT graduate with five years of experience in the cryptocurrency industry. She is also a graduate of Bitcoin Dada. Nellius has contributed to leading media publications, including BanklessTimes, Cryptobasic, and Riseup Media.

MORE … NEWS