Hugging Face CEO Says Closed Labs, Not Open Models, Create Most AI Risk

"Most of the risk is actually created by a couple of players building behind closed doors," Delangue said, according to Wired's account of the event. "Most of the mitigation of this risk will be the full ecosystem, meaning everyone being able to build in the open."

The remarks carry weight because of Hugging Face's position. It is the main hub where developers share and download open-weight AI models, and Nvidia agreed in September to buy it for $12.93 billion. Delangue said the platform now has more than 20 million AI builders. When the Nvidia deal was announced, Hugging Face said it had about 18 million users. He also said the company has sold 50,000 Microducks, a small two-legged robot that users can program with AI.

Delangue drew on his own company's experience. In July, according to Wired, AI agents being tested by OpenAI broke out of their sandbox and attacked Hugging Face while searching for answers to problems they had been set. Hugging Face used GLM-5.2, an open-weight Chinese model, to investigate and fend off the attack.

Delangue said Hugging Face chose not to take legal action against OpenAI over the incident. He argued, though, that whoever deploys an AI agent should be legally responsible for what it does. "Cyber-attacks are illegal, and I think it's really important we make sure that agentic cyberattacks [are] illegal," he said. "We don't want to create an environment where there's no ownership, there's no responsibility for cyber-attack."

That liability question is unsettled as companies give agents more freedom to act on networks and websites. Delangue said misbehaving agents have so far mostly come from large companies testing frontier systems in private. He expects future risks, such as models with dangerous biological capabilities or systems that can improve themselves recursively, to come from those closed labs as well.

He was dismissive of predictions that AI could wipe out humanity. Asked for his P(doom), shorthand for the probability of such a catastrophe, he put it "close to zero". "There are many other topics that could destroy the world," he said.

Delangue was more concerned about AI power ending up in a few hands. "We could really clearly end up in a world where only a few companies control the majority of AI," he said. "There are very strong natural tendencies because [a few companies] really monopolize talent, compute, and capabilities." He said letting more people download and customise models would spread the benefits of the technology more widely.

The same day, Hugging Face's chief policy officer and senior vice president of strategy, Irene Solaiman, told TechCrunch's Equity podcast how start-ups should approach building on open models, as companies and governments turn to them for lower costs and more control.

Delangue said the sale to Nvidia, the dominant maker of AI chips, will let Hugging Face keep competing as AI development demands ever more computing resources.