Arcee, a US open source AI lab, says Chinese models are not inherently dangerous
As Chinese AI models grow in capability and popularity among U.S. companies, the arguing over what should be done about them has reached a fever pitch.
As Chinese open-weight AI models grow in capability and popularity , arguments about what should be done about them have once again reached a fever pitch.
There’s talk that the Trump administration might try to ban them (though it hasn’t yet acted on the idea). Meanwhile, proprietary model makers, particularly OpenAI and Anthropic, appear increasingly concerned about them.
Open-weight models such as Moonshot AI’s Kimi K3 or Alibaba’s Qwen offer inference at a fraction of the token cost of closed source models from these large U.S. labs. The fear is that they also pose some sort of threat. Certainly they threaten the profit margins of the large proprietary AI labs.
But should enterprises running these models in their own data centers succumb to the fear that they could be a vector for Chinese hackers?
No, says Lucas Atkins, the CTO of Arcee , which is building open models to give U.S. companies a homegrown alternative to Chinese models.
If any startup would benefit from a ban on Chinese models, Arcee would. But Atkins says China’s open models are no more dangerous than any other open source software a company may use. In fact, he says, they even offer benefits even to his own company.
“A lot of people view this as similar to a Chinese software program. Like, it was coded with these x, y, z intentions” that a bad actor could simply command, he said.
“That is fundamentally not how these models are trained. There is really not any way for an Arcee, or an Alibaba, to make a model, have someone run it in their own environment and for us have any access to it whatsoever,” he explained.
While most of these models are what’s known as “open weight” and are not really fully open source software, the source code (the part that will actually run on servers), if it is downloaded from open source sites like Hugging Face, is similarly largely visible and reviewable. (What isn’t available is the methods and data used to train the models.)
Large organizations should put any model core through their security testing and inspection processes, and they will also often post-train the models for their specific uses and can examine areas like bias, toxicity, hallucinations, and sensitivity to certain topics. So they work with, optimize, and understand the models before people start sending them prompts.
Could a model that is used for coding somehow throw malicious backdoors into the code it writes? Again, while that’s theoretically possible, it would require acrobatic feats to accomplish.
“There’s no reason that a sophisticated enough actor couldn’t train a model to be a completely amazing coding model in every circumstance, but when presented with a certain type of code base … some hidden training would kick in,” Atkins, who spends his days training models, postulated. But he adds: “I don’t know how you would do this.”
Because large language models are by nature creative, the odds are slim of getting a contemporary model to spit out malware in response to a preplanned perfect storm of context and prompt. Even slimmer are the chances that any enterprise would then use that code.
Could it happen in the future? That’s anyone’s guess. But enterprises are also building their AI apps to be model-agnostic and to use multiple models. So even if Chinese models are the best for the price today, enterprises won’t be locked into using them forever.
“I think instead of the conversation being about how to ban Chinese models, it should be about how do we foster a good, open ecosystem here in the U.S.,” Atkins says.
Arcee also gains advantages from Chinese models. Because they are open, the startup “benefits from those models being good because we can learn what they did. We can build on top of them. Then they can learn what we do,” he says. “We have tremendous respect for the people building those models, the individual researchers.”
Ultimately, the way to compete with Chinese models “is to release a model that is better,” says Atkins. “We need to give them something to talk about.”
When you purchase through links in our articles, we may earn a small commission . This doesn’t affect our editorial independence.
Scale faster. Grow your portfolio. Gain practical expertise. No matter your goal, Disrupt can empower you. Save up to $330 toda y!
Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents Amanda Silberling
Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents
Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents
Light made a flip phone — it’s colorful and it’s cheap Amanda Silberling
Light made a flip phone — it’s colorful and it’s cheap
Light made a flip phone — it’s colorful and it’s cheap
AI music generator Suno breach affects 55M users, per Have I Been Pwned Zack Whittaker
AI music generator Suno breach affects 55M users, per Have I Been Pwned
AI music generator Suno breach affects 55M users, per Have I Been Pwned
Anthropic’s landmark $1.5B copyright settlement is approved Kirsten Korosec
Anthropic’s landmark $1.5B copyright settlement is approved
Anthropic’s landmark $1.5B copyright settlement is approved
Google is working on a new AI chip designed to make Gemini more efficient Lucas Ropek
Google is working on a new AI chip designed to make Gemini more efficient
Google is working on a new AI chip designed to make Gemini more efficient
X relaunches a rebuilt Android app after year-long effort Sarah Perez
X relaunches a rebuilt Android app after year-long effort
X relaunches a rebuilt Android app after year-long effort
Judge pauses $110B Paramount-Warner Bros. merger Aisha Malik
Judge pauses $110B Paramount-Warner Bros. merger
Judge pauses $110B Paramount-Warner Bros. merger
Key takeaways
- The growing popularity of Chinese AI models may impact the Brazilian market, necessitating critical analysis of security and privacy.
- Implementing robust security practices is more important than the origin of the AI models used by companies.
- The ability to audit and understand AI models can be a competitive advantage in the saturated technology market.
Editorial analysis
The discussion around the safety of Chinese AI models, especially in the context of their growing popularity and capabilities, is relevant not only for the U.S. but also for Brazil. The country, which is establishing itself as a technology innovation hub, should closely monitor how the competition between open and closed-source models unfolds. Arcee's stance that open models are not inherently dangerous could serve as a starting point for similar debates in Brazil, where AI adoption is on the rise.
Moreover, concerns about cybersecurity and data protection are critical issues for Brazilian companies considering the implementation of AI models. The assertion that open models can be as secure as any other open-source software suggests that companies should focus more on implementing robust security practices rather than the origin of the model itself. This could lead to increased demand for AI solutions that are transparent and auditable, fostering a healthier and more innovative ecosystem.
In the current landscape, it is essential for Brazilian companies not only to adopt AI technologies but also to engage in discussions about the ethics and safety of these technologies. The ability to audit and understand the AI models they use can be a competitive advantage, especially in an increasingly saturated market. Companies must be aware of the security and privacy implications when integrating AI into their operations, ensuring they are prepared to mitigate potential risks.
Finally, what we observe is a growing trend of collaboration and development of local solutions that meet the specific needs of the Brazilian market. The ability to develop local alternatives to foreign models can not only reduce dependence on external technologies but also foster innovation and competitiveness in Brazil's technology sector.
What this coverage includes
- Clear source attribution and link to the original publication.
- Editorial framing about relevance, impact, and likely next developments.
- Review for readability, context, and duplication before publication.
Original source:
TechCrunch AIAbout this article
This article was curated and published by AIDaily as part of our editorial coverage of artificial intelligence developments. The content is based on the original source cited below, enriched with editorial context and analysis. Automated tools may assist with translation and initial structuring, but publication decisions, factual review, and contextual framing remain editorial responsibilities.
Learn more about our editorial process