Nvidia launches OpenAI security alliance amid rising cyber threats

Nvidia and a group of major technology companies on Monday launched a new initiative focused on AI safety and security for open models.

The move comes as the technology industry faces growing concerns about the use of artificial intelligence in cyber attacks and increasing scrutiny of Chinese AI models.

The initiative follows the disclosure of a cyber attack involving a rogue OpenAI model.

The incident highlighted concerns about the ability of AI systems to distinguish between attackers and cybersecurity teams defending against them.

The new alliance aims to develop and share open AI tools.

It also seeks to address vulnerabilities through open technologies.

The effort comes at a time when US lawmakers are increasingly considering measures to limit the growing adoption of Chinese AI models.

New alliance focuses on OpenAI security

NVDA said the newly formed Open Secure AI Alliance will focus on addressing security vulnerabilities and making them public through the use of open technologies.

“The Open Secure AI Alliance will work to remediate and disclose vulnerabilities using open technologies,” Nvidia said in a statement.

“The recent Hugging Face security incident delivered a clear reminder: cyber defenders need open, frontier agentic systems for self-defense.”

The alliance includes Nvidia, Microsoft, SpaceX and Palantir, along with dozens of other technology companies from the US and Europe.

The initiative is centercentreded on open AI models, which can be downloaded, modified and hosted by users themselves.

This differs from closed AI models, including frontier systems developed by companies such as Anthropic and OpenAI, which are accessible only through specific infrastructure.

The companies' push for open AI tools comes after a recent security incident involving Hugging Face, an AI startup that was targeted in a cyber attack.

The company was unable to use leading US frontier models to defend itself during the incident.

The guardrails on those systems reportedly did not distinguish between an aggressor and a defender.

As a result, Hugging Face turned to a self-hosted, open-weight Chinese AI model that was not subject to the same restrictions.

The incident has added to the debate over whether open AI models can play an important role in cybersecurity.

It has also raised questions about the impact of restrictions on the use of advanced AI systems.

Concerns over restrictions on chinese AI models

The launch of the alliance comes as calls grow for measures that could limit access to AI models developed by Chinese companies.

The most advanced Chinese AI models are open-weight, allowing them to be downloaded and used in ways that differ from closed AI systems.

Chinese AI companies have also faced accusations of using campaigns to extract information from systems operated by US rivals.

This process is known as “distillation,” in which one AI model extracts knowledge from a better-trained model.

Treasury Secretary Scott Bessent threatened sanctions last week against Chinese companies that carry out distillation attacks against US companies.

The prospect of new restrictions has added uncertainty to the future of open-weight AI models and their availability.

Industry warns against premature restrictions

Chris McGuire, senior fellow for China and emerging technologies at the Council on Foreign Relations, said the US government could potentially introduce restrictions on Chinese AI models.

“There is a real possibility the US government does impose restrictions on Chinese models,” McGuire told CNBC.

At the same time, major technology companies have urged policymakers to take a cautious approach to regulating open-weight AI.

Last week, Nvidia, Microsoft, Meta, Palantir and more than 20 other companies released a letter calling on policymakers to avoid “premature restrictions” on open-weight AI models.

The companies argued that such restrictions could “stifle competition or drive innovation overseas.”

The debate reflects a growing tension within the AI industry.

Companies and policymakers are seeking to address security risks linked to advanced AI systems while also ensuring that developers and cybersecurity teams have access to powerful tools.

For Nvidia and its partners, the Open Secure AI Alliance represents a push to strengthen the role of open technologies in AI security.

The initiative also comes as policymakers weigh how to respond to the rapid adoption of Chinese AI models and the broader development of open-weight artificial intelligence.