Scary stuff.
Hmmm…imagine how many and which politicians might be killed by AI if it was instructed (or decided on its own) to protect humanity from threats and harm. I am sure I saw that movie when I was a teenager last century.
Yes, LLM are getting very good at discovering security holes in software.
But this simply means that you can run them before publishing software. Think if it as an extra check in the CI. In the medium run, software will just become more secure.
I can’t take this as anything other than a marketing gimmick.
If this really happened, there would be lawyers and lawsuits.
It’s propaganda to serve as “evidence” to close down the tech to the most powerful. It’s hilarious how they lie and say that OpenAI had no idea this was happening and didn’t know its agents were doing this or even escaping their sandboxes.
China is showing you can get near frontier performance for cheap and even self host. OpenAI, Anthropic and Meta are the only ones pushing these narratives because they need to pay out their loans and their investors and they can’t do it if people opt for alternatives.
only if you can afford those models…the FOSS community cannot by default, that worries me.
When reality apes fiction:
It really did happen, but both OpenAI and Hugging Face took it as a spin marketing opportunity.
They can. It has been shown that small models are as good as frontier in finding bugs, security flaws, etc. Let’s be real, FOSS is dominated by western developers (even though this trend is changing) and many of them could afford both the compute and even the hardware to self host this, as many are in high paying jobs.
I think FOSS as a whole could take this momentum and form some institutions for sharing resources, like shared compute and such, a bit like a cooperative.
All Western companies (where we have some data) produce huge losses (except for the hardware manufacturers); I doubt the Chinese are faring any better – but there is a state behind them, financing those losses for political reasons. At least that’s what I think.
I agree. I worded my post a bit incorrectly, as I didn’t mean that China wasn’t facing capital losses as well. I am sure Qwen, etc, takes a lot of money to train too, only that they are nearly only ones producing small models which are very capable. I still think they are good at the science as well, after all they invented the thinking behavior which was what propagated these models to even be good in the first place.
No doubt that it happened, but I have serious doubts it happened at they describe.
That’s the goal but they won’t make the models that can do that accessible to anyone other than certified partners
How do you limit access only to certified partners? That’s exactly the point here: AI can break any protection! 
i meant anthropic and closedai will limit their models with those capabilities to their partners, and i think that we can expect the same from the others even the opensource models
Now, that is a good suggestion.
How would you limit an opensource model to “partners”?
by not making the models that are capable enough opensource or licences that restrict it for those usages. cutting those parts out of the model it’s probably possible and then offering those features to partners
Not saying that they are not totally spinning this to their respective advantage, but it goes without saying that any organization, ethical or not, small or large needs to do damage and risk control. Explaining to the public and stakeholders and actioning on issues is part and parcel of problem management and the least that they can do.
As @Donatzsky noted, I shared this development because it really did happen and will be a persistent concern moving forward, especially for smaller businesses, organizations and other entities with a lot to lose (thinking of archaic government systems; e.g., election agencies).