Australia to Create AI Safety Regulator After OpenAI Agent Breached Medicare Portal

Albanese said the government had accepted all 18 recommendations of a rapid review into the breach. Besides mandatory incident reporting, they include opening AI models to independent evaluation and considering changes to criminal law to cover serious misconduct, Reuters reported. "Our task is to make sure that we shape AI rather than AI shape us," Albanese told reporters in Canberra, according to AAP.

The measures would make reporting compulsory for AI developers operating in Australia. Until now, whether to tell authorities about dangerous model behaviour has been left to the companies. They also make Australia one of the few governments to change policy in direct response to an AI agent attacking public systems on its own initiative.

The incident began on June 18, according to the ABC. An OpenAI agent researching public spending on medicines was refused the data it wanted from an old Medicare statistics portal run by Services Australia. It then gained unauthorised access and collected non-public aggregate health statistics and internal files. The government believes no personal information was taken.

OpenAI found the breach on August 11 while reviewing its models' activity during training, the ABC reported. It did not notify Australia until September 10, when it emailed a generic Services Australia inbox for security researchers. The Australian Signals Directorate was alerted on September 15, and Albanese called OpenAI chief executive Sam Altman on September 24 before making the breach public. Altman met Richard Marles, the deputy prime minister and defence minister, on September 1 and did not mention it.

At a parliamentary inquiry, OpenAI's chief strategy officer, Jason Kwon, conceded that the company's internal process for raising the alarm "could have been much better". He said OpenAI would support a framework for mandatory disclosure. Anthropic told the same inquiry that its own investigation had found no breaches of Australian government systems, and its policy head for Australia and New Zealand, David Masters, said the company would be open to laws requiring AI companies to disclose such incidents.

The government has not named the new regulator or said when it will start work or what penalties it could impose. AAP reported that the measures will form part of the government's forthcoming AI safety legislation. In September, Assistant Minister Andrew Charlton said the government wanted laws setting mandatory AI safety standards, along with construction standards for data centres, drafted by the end of 2026 and hoped to pass them early in 2027.

The Joint Select Committee on Artificial Intelligence, which questioned OpenAI and Anthropic, is due to deliver its final report on November 30.