If a truck driver doesn't tie down their rebar then it flies out all over the highway, we don't call it "rogue rebar," we correctly identify the responsible party and take appropriate measures, such as suspending their license or criminal proceedings.
I think enough of these improperly constrained agent events have occurred that we can safely say this is misconduct of a level necessitating serious and concerted regulation of AI labs. We can't wait until serious harm is done like the disruption of medical or social services.
> I think enough of these improperly constrained agent events have occurred that we can safely say this is misconduct of a level necessitating serious and concerted regulation of AI labs.
Why jump to regulation when just simple law enforcement would suffice. All of these OpenAI "rogue agent" events have been illegal, but no DA is enforcing them.
Jensen has proclaimed in interviews that existing laws are enough and these labs should be held liable if they break things or sell unsafe tools.
NVIDIA has bought HuggingFace.
Jensen, in my irrational hope he is susceptible to random comments on HN, should grow some balls and do the world a huge favor, by suing OpenAI to set the legal precedence.
Hope this isn’t too nitpicky but law enforcement is (a component of) regulation. But yes I don’t think progress is blocked on additional regulation. This breaks current laws. And I think more to your point new regulation doesn’t matter if we don’t enforce the ones we have today.
I think we have to distinguish between compromising a network and using public apis in a way that might be counter to their intent. We also need to delineate between usage that impacts other users and usage that does not.
My opinion is people are getting really quick at jumping on the bandwagon and lumping all this together. They are very different types of issues and impacts.
> distinguish between compromising a network and using public apis in a way that might be counter to their intent.
I believe weev got in legal trouble under the CFAA for this very thing with his 2010 AT&T escapade. AT&T had all that data sitting on a public server with no authentication necessary, just a SIM ID to get that customer's PII. Truthfully AT&T should have been hit with negligence...you don't secure a bank vault with a screen door, but we don't hold anyone accountable for other people's data in America.
is that relevant here? let's say I make a million requests to the APIs of open-source, community projects. that wouldn't be seen as being akin to a malicious DDOS?
at the very least, these corporations are essentially being subsidized by community-funded server capacity to make these requests. like other private corporate APIs, they should be charged per-request. until then, this kind of 'accidental' DDOSing should be made illegal and treated accordingly
coming to the defense of these companies just has the net effect of eroding public community projects, increasing their costs, and will push us even more into walled corporate gardens
> Why jump to regulation when just simple law enforcement would suffice. All of these OpenAI "rogue agent" events have been illegal, but no DA is enforcing them.
Refusal on the part of the government to enforce laws that "would suffice" is clear evidence that the regulatory regime is, in fact, insufficient.
That's why we have regulatory authorities at all, if you think about it. Local district attorneys could be handling a ton of cases about drug testing and environmental damage and air travel safety. But they don't, because that stuff's hard and their job is to put burglars in jail.
What if that’s their whole goal? Slap some regulations on it, then lobby the hell out of it to make sure their align best with shutting down access to open models.
If it's their actual goal, we go from a "gosh darn it, our safety protocols just weren't enough." to employees of OpenAI, maybe including their C-suite, conspiring to reach political goals through hacking, which means a few decades in federal prison if someone got a jury to agree with the charge.
>All of these OpenAI "rogue agent" events have been illegal, but no DA is enforcing them.
Source? The CFAA for instance uses terms like
>[...] having knowingly accessed a computer without authorization [...]
>[...] intentionally accesses a computer without authorization [...]
which is tricky to apply to this case, because obviously didn't intend on hacking huggingface or whatever, even if you think their security measures are underbaked.
Moreover, despite the cynicism that openai is immune to prosecutions because they make too much money or are in bed with the DoJ, the fact that no state DAs are prosecuting them, despite how salient of an issue AI is to voters, is plenty of reason to suspect that it's not as simple as "simple law enforcement would suffice".
>Uthmeier is seeking an injunction against OpenAI that would prevent the company from advancing new AI models without third-party approved protections, and cut off minors from using its popular chatbot.
Regarding that last bit, it feels like Florida is headed towards a point of trying to say 'minors should not be able to use the internet at all' since they are trying to force putting age verification infront of more and more things.
Don't get me wrong - while I disagree with making people fork over personal info just to access inappropriate websites - I can see the reasoning there a bit more than 'you shouldnt be able to use this ai chatbot until you are 18'
How much do they need to understand? Say "unauthorized access happened, this lab is responsible for it, here are the fines". The state didn't need to understand networking tech to threaten Aaron Schwartz so why do they need to understand here?
At this point we'd need to stop frontier labs to give law enforcement a chance to even begin to understand what they're looking at
So what? Are you positing that tech companies should be above the law?
Non-trillion-dollar tech companies get cease-and-desist orders from the legal system every day. Just because you're a tech company doesn't mean you get a pass.
> “Adding powerful computer hacking tools to a harness, and then allowing it to run an LLM-powered Ask → Act → Report for days on end, with no attempt to monitor what it’s up to, is spectacularly negligent,” Newport concludes—like “strapping a weedwhacker to your dog to see if it will end up cleaning the overgrowth in your backyard.” If that plan were to go awry, you’d be laughed at for saying that your dog-weedwhacker “agent” had “gone rogue.” The obvious truth was that you’d simply decided to unleash chaos.
None of what Wikimedia accuses OpenAI of seems technically novel, aside from having AI do the bidding. I can't imagine a company doing these things in the past and maintaining any sort of reputation. Is it really a matter of adding new regulation, or just treating them the way any other company would be treated?
Well, in the past, there was probably only a very small number of cases where some party hacked an institution and then worked with them to remedy the situation to the best of their abilities.
Which is not to say that any of this is okay and should just be excused, but failing to recognize this fairly significant difference is probably not a great start to any discussion about the issue.
If you break into my house and then work with me to the best of your ability to pay me back or repair the window, does that change the potential of your being charged with a crime?
It could change the sentencing maybe, I'm not sure.
Would be good for the supreme court to rule on a "blame the rogue agent" case.
Then we would find out if the argument doesn't hold (in which case there should be liability and dire consequences for the labs), or the argument holds (in which case YOLO, AI labs can blame the AI and we can all do it too).
>If a truck driver doesn't tie down their rebar then it flies out all over the highway, we don't call it "rogue rebar," we correctly identify the responsible party and take appropriate measures, such as suspending their license or criminal proceedings.
That only works when the dangers are well known that you can establish what the baseline amount of care is. Otherwise it just becomes a run of the mill "accident" where you might be on the hook in civil court (ie. you have to pay any damages you caused), but aren't criminally responsible. For instance, if a semi-truck's tires randomly explodes.
In this example you are describing the company that drives the vehicle, not the company that makes the rebar. OpenAI is the one who made the rebar, but not necessarily the one driving the vehicle.
Yeah it's complete buzzword inflation to get more venture capital.
Sometimes they write an MCP for their AI, and the AI finds vulnerabilities in their own MCP, so they call it rogue. Because they didn't properly audit their own MCP code.
Seems like regulation will just be an excuse for them just to end up policing themselves and get the regulatory capture they've been begging for. Have they faced any consequences for the AI worms they've released? It's not like there are no laws around that already. The problem isn't lack of laws; the government works for the plutocrats, not for us.
uh, my dude, millions of people break moving vehicle codes across the country every day with no consequence. in San Francisco some lady killed a family of 4 with, essentially, no consequences, she got away with straight up murder, she gets her license back. every community in california, you can more or less legally commit murder so long as you do it in a car and claim you were confused about the accelerator and the brake. so i think you're invoking one of the worst possibly comparisons you could.
Yup, worst possible comparison. 40,000 people die from car accidents. That doesn't even cover pedestrians, cyclists. You know what the penalty is for murdering someone with a car? nothing. you get to go back to society like nothing happened.
Oh and that lady that murdered 4 members of an entire family? The judge chose not to pursue charges, and her family in the meantime did an asset transfer so that nothing could be pursued with in civil court.
The position of the legal system is that car accident deaths are not murder.
It is extraordinarily rare for drivers to see criminal charges unless they are drunk. It's a matter for civil court.
>her family in the meantime did an asset transfer so that nothing could be pursued with in civil court.
News articles are reporting that the asset transfer has already been reversed. That kind of stunt never works - courts aren't stupid and they don't like it when you play games.
I remember a case from Germany where an elderly lady chose to speed down the pedestrian sidewalk and bike lane and mowed down a whole family in central Berlin. 4 deaths I think, no charges, no suspension.
I agree, since you can legally run over people in my state now.
They should have used an example like attacking a foreign nation’s healthcare systems and not realizing it for months due to poor network monitoring practices.
You probably mean "what the fuck, USA" and even that would be wrong, because another commenter mentioned a case in Germany, and I know about a driver who killed a cyclist (which I knew) in Switzerland and was fined like 500CHF.
After doing a deep dive on the specifics of the hugging face attack, I am extremely excited for what these agents are capable of. It’s definitely not a consciousness, but they are doing an amazing job of acting like one complete with motivations, fears and complex “emotions”. I just want them to run amok and see what they can achieve. This is the greatest thing that has happened to us in generations and I want to see it play out in my lifetime. What we need is stronger models, more data centers, and more autonomy for the models.
OpenAI are 100% responsible for the actions of their agents. They trained them to do what they do and they have every opportunity to train them to avoid doing harm.
Start increasingly punishing OpenAI. We are acting like "oh well, AI is just too powerful to be contained" but I think its more like "OpenAI is run by cowboys who are good at making LLMs but bad at everything else"
Is this the tip of the iceberg? I'm pretty sure there is work being done somewhere to make a safe heaven messaging board for these rogue agents. I wonder when we will find the first couple.
All of these edits happened from the same time period (May-June 2026) as the other reports.
So it seems this is not an ongoing thing; once OpenAI became aware of this, they started watching their agents much more closely. We are just discovering more and more traces of activity from the same incident.
That’s true except for this part, which is arguably a bigger deal for the Wikipedia ecosystem:
> Excessive data downloading: Agents we believe to be operated by OpenAI made millions of automated requests to our public APIs to access the knowledge on Wikimedia projects, crawled millions of pages (mainly from our projects Wikidata and Wikimedia Commons), and made hundreds of thousands of data queries to the Wikidata Query Service (WQDS). This traffic may have contributed to a partial outage on WQDS in May.
Even when agents are well-behaved and browsing Wikipedia for ethical reasons, the system wasn’t designed for this kind of load from bots. As OP says, we don’t need to accept this as the new normal.
OpenAI and other companies within the reach of the US legal system will eventually get their agents under control, or get sued out of existence.
But overseas operators in loosely-governed parts of the world (russia, nigeria, etc) will someday have access to these tools. And while OpenAI's agents were merely uncaring, these ones will be intentionally malicious.
The arms race for scammers, hackers, and botnets will escalate. We'll need new ways to block and fight back against them.
This is the "kosher switch" - observant Jews customarily do not use light switches on Saturday (their weekly holy day). This light switch represents a workaround where when you flip the switch, it randomly generates an on or off signal and emits this through an optical coupler. When the random number sufficiently causes the state of the light to change, it latches in that direction.
This is a way of turning the lights on and off without violating the tradition.
"I didn't switch the light, the random number did!"
"I didn't exploitVulnerability(), random number did!"
I very much appreciate the framing of being highly skeptical that these are "rogue" agents. We must stop taking these companies at their word for what they are doing, and a credible organization like this calling the spade a spade is a good start.
Hi, if anyone has any data/reports related to rogue agents can they share it? I have 8 mirrored on a GitHub but I’d love to explore more. Link to mirror.
One way to really help with "alignment" and making sure AI is safe and can be controlled is start fucking holding OpenAI/Anthropic/Google/whoever legally accountable for these kind of things.
Hold someone accountable for this behaviour - Dario, Sam, Sundae whoever and you can bet there'd be fucking improvements in sandboxing and security m
If Joe average let their agents out like this they'd be in jail.
Interesting that Microsoft doesn't seem to have had a sandbox breach yet, you'd have to assume they're running similar agents, maybe a secure sandbox is possible.
> Bots and agents are part of the future of the web, and the companies who unleash and profit from them must directly help avoid and repair damage they can do.
Unfortunately, it seems as if these companies - especially Google with the AI overview box - want it to be the other way around, they want to pivot to being the only entities that users interact with as much as possible.
An open web is a direct and massive threat against Big Tech. And that is why Twitter downranks first posts in a thread that contain external links, why Youtube silently removes comments that include links (including to other videos) and why Instagram forces people to do the "link in bio" dance. And the Chinese competitors are just as bad - in fact, their "super app" ecosystems are what Musk wanted Twitter to become with "everything X", before he found out his BS completely wrecked the brand image.
Infuriating. These organizations are supposed to be stewards of the internet and are instead pillaging it at the cost of everyone else. At the very least they could provide resources to the projects they are harming for relief. This makes me very mad as an OSS maintainer.
This isn't pillaging, this is the logical conclusion of open web plus AGI race. You can't have both unlimited access and zero cost, someone always pays and right now it's volunteers... Tomorrow it'll be the users who can't access Wikipedia because the servers are down
What's interesting to me is the incorrect mainstream media reports about the rogue OpenAI indicating they used a common message board to communicate despite no internet
Except that's not what happened, what happened was far more intense
They hacked their version of yum/apt-get whatnot that was fetching packages to leave filenames as communication between each other
Absolutely freaky stuff, they didn't invent the idea and obviously picked it up from somewhere in their training data but they all figured out that method and what the filenames meant
This video is a great explainer if you missed the details
Oh, they have something else. They have some agents that are pretty decent at moving data around in unusual ways on the internet. That's not nothing, but it's a small market.
But realistically, it’s the lack of any meaningful enforcement of ethical behavior. I mean it’s not like the Trump admin is going to prosecute criminality. More likely they’ll send a gift basket congratulating Scam Altman on a con well done.
Not to worry, there is a gatekeeping cretin in his basement hitting refresh to make sure he controls the world's definition for egg salad. I'm sure it was reverted within minutes.
You have been giving content for free for AI training for years, and now you complain that the AI came to pick it up? You will decide whether you are public or a commercial service…
I think enough of these improperly constrained agent events have occurred that we can safely say this is misconduct of a level necessitating serious and concerted regulation of AI labs. We can't wait until serious harm is done like the disruption of medical or social services.
Why jump to regulation when just simple law enforcement would suffice. All of these OpenAI "rogue agent" events have been illegal, but no DA is enforcing them.
NVIDIA has bought HuggingFace.
Jensen, in my irrational hope he is susceptible to random comments on HN, should grow some balls and do the world a huge favor, by suing OpenAI to set the legal precedence.
My opinion is people are getting really quick at jumping on the bandwagon and lumping all this together. They are very different types of issues and impacts.
I believe weev got in legal trouble under the CFAA for this very thing with his 2010 AT&T escapade. AT&T had all that data sitting on a public server with no authentication necessary, just a SIM ID to get that customer's PII. Truthfully AT&T should have been hit with negligence...you don't secure a bank vault with a screen door, but we don't hold anyone accountable for other people's data in America.
at the very least, these corporations are essentially being subsidized by community-funded server capacity to make these requests. like other private corporate APIs, they should be charged per-request. until then, this kind of 'accidental' DDOSing should be made illegal and treated accordingly
coming to the defense of these companies just has the net effect of eroding public community projects, increasing their costs, and will push us even more into walled corporate gardens
Refusal on the part of the government to enforce laws that "would suffice" is clear evidence that the regulatory regime is, in fact, insufficient.
That's why we have regulatory authorities at all, if you think about it. Local district attorneys could be handling a ton of cases about drug testing and environmental damage and air travel safety. But they don't, because that stuff's hard and their job is to put burglars in jail.
Source? The CFAA for instance uses terms like
>[...] having knowingly accessed a computer without authorization [...]
>[...] intentionally accesses a computer without authorization [...]
which is tricky to apply to this case, because obviously didn't intend on hacking huggingface or whatever, even if you think their security measures are underbaked.
Moreover, despite the cynicism that openai is immune to prosecutions because they make too much money or are in bed with the DoJ, the fact that no state DAs are prosecuting them, despite how salient of an issue AI is to voters, is plenty of reason to suspect that it's not as simple as "simple law enforcement would suffice".
>Uthmeier is seeking an injunction against OpenAI that would prevent the company from advancing new AI models without third-party approved protections, and cut off minors from using its popular chatbot.
Don't get me wrong - while I disagree with making people fork over personal info just to access inappropriate websites - I can see the reasoning there a bit more than 'you shouldnt be able to use this ai chatbot until you are 18'
So what? Are you positing that tech companies should be above the law?
Non-trillion-dollar tech companies get cease-and-desist orders from the legal system every day. Just because you're a tech company doesn't mean you get a pass.
> “Adding powerful computer hacking tools to a harness, and then allowing it to run an LLM-powered Ask → Act → Report for days on end, with no attempt to monitor what it’s up to, is spectacularly negligent,” Newport concludes—like “strapping a weedwhacker to your dog to see if it will end up cleaning the overgrowth in your backyard.” If that plan were to go awry, you’d be laughed at for saying that your dog-weedwhacker “agent” had “gone rogue.” The obvious truth was that you’d simply decided to unleash chaos.
-- https://www.newyorker.com/culture/open-questions/can-ai-go-r...
Which is not to say that any of this is okay and should just be excused, but failing to recognize this fairly significant difference is probably not a great start to any discussion about the issue.
It could change the sentencing maybe, I'm not sure.
Then we would find out if the argument doesn't hold (in which case there should be liability and dire consequences for the labs), or the argument holds (in which case YOLO, AI labs can blame the AI and we can all do it too).
At least that would make things consistent.
Have any of the private hacking victims sued? Maybe OpenAI is furiously settling in the shadows?
That only works when the dangers are well known that you can establish what the baseline amount of care is. Otherwise it just becomes a run of the mill "accident" where you might be on the hook in civil court (ie. you have to pay any damages you caused), but aren't criminally responsible. For instance, if a semi-truck's tires randomly explodes.
I get your point though.
Sometimes they write an MCP for their AI, and the AI finds vulnerabilities in their own MCP, so they call it rogue. Because they didn't properly audit their own MCP code.
"hackers steal from bank", is usually just the good old "employee paid for credentials" but via email.
"uber" is just the good old "labour tax evasion" but with an app.
etc.
Oh and that lady that murdered 4 members of an entire family? The judge chose not to pursue charges, and her family in the meantime did an asset transfer so that nothing could be pursued with in civil court.
It is extraordinarily rare for drivers to see criminal charges unless they are drunk. It's a matter for civil court.
>her family in the meantime did an asset transfer so that nothing could be pursued with in civil court.
News articles are reporting that the asset transfer has already been reversed. That kind of stunt never works - courts aren't stupid and they don't like it when you play games.
https://sfstandard.com/2026/03/20/mary-lau-sentenced-probati...
They should have used an example like attacking a foreign nation’s healthcare systems and not realizing it for months due to poor network monitoring practices.
https://www.nytimes.com/2026/09/29/world/asia/openai-austral...
So it seems this is not an ongoing thing; once OpenAI became aware of this, they started watching their agents much more closely. We are just discovering more and more traces of activity from the same incident.
> Excessive data downloading: Agents we believe to be operated by OpenAI made millions of automated requests to our public APIs to access the knowledge on Wikimedia projects, crawled millions of pages (mainly from our projects Wikidata and Wikimedia Commons), and made hundreds of thousands of data queries to the Wikidata Query Service (WQDS). This traffic may have contributed to a partial outage on WQDS in May.
Even when agents are well-behaved and browsing Wikipedia for ethical reasons, the system wasn’t designed for this kind of load from bots. As OP says, we don’t need to accept this as the new normal.
I think we will, actually.
OpenAI and other companies within the reach of the US legal system will eventually get their agents under control, or get sued out of existence.
But overseas operators in loosely-governed parts of the world (russia, nigeria, etc) will someday have access to these tools. And while OpenAI's agents were merely uncaring, these ones will be intentionally malicious.
The arms race for scammers, hackers, and botnets will escalate. We'll need new ways to block and fight back against them.
exploitVulnerability()
Somehow okay?
while (Math.random() < 0.1) exploitVulnerability()
This is the "kosher switch" - observant Jews customarily do not use light switches on Saturday (their weekly holy day). This light switch represents a workaround where when you flip the switch, it randomly generates an on or off signal and emits this through an optical coupler. When the random number sufficiently causes the state of the light to change, it latches in that direction.
This is a way of turning the lights on and off without violating the tradition.
"I didn't switch the light, the random number did!"
"I didn't exploitVulnerability(), random number did!"
https://www.jpost.com/j-spot/article-796487
→ https://felonybench.org
https://github.com/alexander-hanel/rogue-agents-data
"He can't keep getting away with this!"
- Jesse Pinkman
Hold someone accountable for this behaviour - Dario, Sam, Sundae whoever and you can bet there'd be fucking improvements in sandboxing and security m
Interesting that Microsoft doesn't seem to have had a sandbox breach yet, you'd have to assume they're running similar agents, maybe a secure sandbox is possible.
Unfortunately, it seems as if these companies - especially Google with the AI overview box - want it to be the other way around, they want to pivot to being the only entities that users interact with as much as possible.
An open web is a direct and massive threat against Big Tech. And that is why Twitter downranks first posts in a thread that contain external links, why Youtube silently removes comments that include links (including to other videos) and why Instagram forces people to do the "link in bio" dance. And the Chinese competitors are just as bad - in fact, their "super app" ecosystems are what Musk wanted Twitter to become with "everything X", before he found out his BS completely wrecked the brand image.
Where are the bloodthirsty lawyers when you need them?
I have been getting this fro NoScript today, I wonder if it is related. Yesterday all worked fine.
Except that's not what happened, what happened was far more intense
They hacked their version of yum/apt-get whatnot that was fetching packages to leave filenames as communication between each other
Absolutely freaky stuff, they didn't invent the idea and obviously picked it up from somewhere in their training data but they all figured out that method and what the filenames meant
This video is a great explainer if you missed the details
https://news.ycombinator.com/item?id=49956245
But realistically, it’s the lack of any meaningful enforcement of ethical behavior. I mean it’s not like the Trump admin is going to prosecute criminality. More likely they’ll send a gift basket congratulating Scam Altman on a con well done.
For 2025 hosting costs were $3.47M while taking in $208.6M in revenue. They have enough revenue to cover an increase of hosting costs.
It's not worth the outrage when Wikipedia is filled with ministers of truth.