Arcee, a US open source AI lab, says Chinese models are not inherently dangerous

As Chinese open-weight AI models grow in capability and popularity , arguments about what should be done about them have once again reached a fever pitch.

There’s talk that the Trump administration might try to ban them (though it hasn’t yet acted on the idea). Meanwhile, proprietary model makers, particularly OpenAI and Anthropic, appear increasingly concerned about them.

Open-weight models such as Moonshot AI’s Kimi K3 or Alibaba’s Qwen offer inference at a fraction of the token cost of closed source models from these large U.S. labs. The fear is that they also pose some sort of threat. Certainly they threaten the profit margins of the large proprietary AI labs.

But should enterprises running these models in their own data centers succumb to the fear that they could be a vector for Chinese hackers?

No, says Lucas Atkins, the CTO of Arcee , which is building open models to give U.S. companies a homegrown alternative to Chinese models.

If any startup would benefit from a ban on Chinese models, Arcee would. But Atkins says China’s open models are no more dangerous than any other open source software a company may use. In fact, he says, they even offer benefits even to his own company.

“A lot of people view this as similar to a Chinese software program. Like, it was coded with these x, y, z intentions” that a bad actor could simply command, he said.

“That is fundamentally not how these models are trained. There is really not any way for an Arcee, or an Alibaba, to make a model, have someone run it in their own environment and for us have any access to it whatsoever,” he explained.

While most of these models are what’s known as “open weight” and are not really fully open source software, the source code (the part that will actually run on servers), if it is downloaded from open source sites like Hugging Face, is similarly largely visible and reviewable. (What isn’t available is the methods and data used to train the models.)

Large organizations should put any model core through their security testing and inspection processes, and they will also often post-train the models for their specific uses and can examine areas like bias, toxicity, hallucinations, and sensitivity to certain topics. So they work with, optimize, and understand the models before people start sending them prompts.

Could a model that is used for coding somehow throw malicious backdoors into the code it writes? Again, while that’s theoretically possible, it would require acrobatic feats to accomplish.

“There’s no reason that a sophisticated enough actor couldn’t train a model to be a completely amazing coding model in every circumstance, but when presented with a certain type of code base … some hidden training would kick in,” Atkins, who spends his days training models, postulated. But he adds: “I don’t know how you would do this.”

Because large language models are by nature creative, the odds are slim of getting a contemporary model to spit out malware in response to a preplanned perfect storm of context and prompt. Even slimmer are the chances that any enterprise would then use that code.

Could it happen in the future? That’s anyone’s guess. But enterprises are also building their AI apps to be model-agnostic and to use multiple models. So even if Chinese models are the best for the price today, enterprises won’t be locked into using them forever.

“I think instead of the conversation being about how to ban Chinese models, it should be about how do we foster a good, open ecosystem here in the U.S.,” Atkins says.

Arcee also gains advantages from Chinese models. Because they are open, the startup “benefits from those models being good because we can learn what they did. We can build on top of them. Then they can learn what we do,” he says. “We have tremendous respect for the people building those models, the individual researchers.”

Ultimately, the way to compete with Chinese models “is to release a model that is better,” says Atkins. “We need to give them something to talk about.”

When you purchase through links in our articles, we may earn a small commission . This doesn’t affect our editorial independence.

Scale faster. Grow your portfolio. Gain practical expertise. No matter your goal, Disrupt can empower you. Save up to $330 toda y!

Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents Amanda Silberling

Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents

Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents

Light made a flip phone — it’s colorful and it’s cheap Amanda Silberling

Light made a flip phone — it’s colorful and it’s cheap

Light made a flip phone — it’s colorful and it’s cheap

AI music generator Suno breach affects 55M users, per Have I Been Pwned Zack Whittaker

AI music generator Suno breach affects 55M users, per Have I Been Pwned

AI music generator Suno breach affects 55M users, per Have I Been Pwned

Anthropic’s landmark $1.5B copyright settlement is approved Kirsten Korosec

Anthropic’s landmark $1.5B copyright settlement is approved

Anthropic’s landmark $1.5B copyright settlement is approved

Google is working on a new AI chip designed to make Gemini more efficient Lucas Ropek

Google is working on a new AI chip designed to make Gemini more efficient

Google is working on a new AI chip designed to make Gemini more efficient

X relaunches a rebuilt Android app after year-long effort Sarah Perez

X relaunches a rebuilt Android app after year-long effort

X relaunches a rebuilt Android app after year-long effort

Judge pauses $110B Paramount-Warner Bros. merger Aisha Malik

Judge pauses $110B Paramount-Warner Bros. merger

Judge pauses $110B Paramount-Warner Bros. merger