Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immediately with a cease and desist. But from the receiving end it's just because the spam is unmanageable.

Hopefully at the end of this decade, a ton of software practices have been overhauled to eliminate classes of problems. Memory-safe language use is a great start - but it’d be great to see innovation in checking for TOCTOU problems, improper/missing authn & authz, and many others.

This is an engineering problem. It won’t be solved by models that “only do dumb shit 1/10th as often, only 0.01% of the time now not 0.1%!” It won’t be solved by adding more models to do even more double-checking before and after the work. It won’t be solved by hoping humans catch it in review. It isn’t solvable by adding outer loops of any sort - though we may get close. To truly solve this will take serious CS research.

I currently have two reports (one RCE on a famous OSS ML platform, one cluster take over on a k8s related projects), both are more than 2 months old without as much as an “F you, get lost”. Just got ignored and ghosted, which hurts a lot, because I spent a lot of time finding, and verifying these (all reports with poc and patch). BUT I understand why it’s happening, because I’m also on the receiving end.

security@ and VDPs have always received BS reports and beg-bounties, but boy oh boy, these days we have two people spending 3-4 days a week sifting through this constant flood of garbage compared to 2-3 tears ago where 1 person could triage the inbox and VDP in a day’s work max, which would’ve been considered very busy. Unfortunately we can’t just shutdown the programs or the mailbox because 1. We do occasionally get important and great stuff that actually matters, and 2. We’re a critical infra company and can’t ignore anything really.

The signal to noise ratio is almost zero, but the “what if” is keeping us swimming through this unending river of garbage and burning us out.

Overall, chaotic mess on both sides.

Ending on a doom-and-gloom note: there will be a reckoning.

(Don’t take the note too seriously though, I’m a SecEng, so I have a built-in doom multiplier lol)

Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to minimize creating bugs in the first place. That could be new or better languages, less 3rd party dependencies, more vetted code, better linters, better fuzzers, whatever. The point the new reality of bugs being easy to find will, actually must, lead to less bugs eventually because the world can't function with easy to find bugs.

> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all.

> Except…

> For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t apply to vulnerability reports. No, vulnerability reports are special. Security researchers are doing us a favor by reporting things confidentially instead of doing full disclosure, so we owe them something, which is not true of regular issues opened on the issue tracker.

[...]

> It’s 2026 and none of the premises are true anymore.

I respectfully disagree.

The premise is absolutely still true: if someone discovers a critical, exploitable vulnerability in your software, the impact and tradeoffs are exactly the same as they were before LLMs started finding bugs. There are just more of them now, so they're easier to come by.

But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.

All of that said, I don't think finding vulnerabilities has really been the difficult security problem for most companies (or open source projects). The difficult problem is dedicating resources to fixing those vulnerabilities instead of building software, products, and/or infrastructure that people want. That problem is absolutely still here today, but I'm optimistic that agentic security developers will be able to take the burden off of development teams in the near future.

For tokens, of course.

We’ve always been able to find heaps, we’ve just never had the right structures to put in the effort and renumerate people for looking (even if they don’t find anything).

One flipside to this is that, because many of these bugs are "shallow" to LLMs, it's actually easier than ever to moderate the worst participants in your vulnerability program -- if someone sends you slop, you can just ban them and wait for the next, better orchestrated LLM to send you a better report for the same vulnerability.

> But give it 1-3 months and the open models will catch up.

I wish that this would stopped being thrown around, what is this timeline based on? How good is your open model from between March and May?

Also, having read "Gödel, Escher, Bach" I know that the hare never catches up with the turtle.

I wonder what the metrics are. Also, not "anyone", just the affordable.

The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now.

Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstration, the communication that was special.

When you realize that this is being written from the perspective of someone who does vulnerability reporting in a professional capacity, you’ll connect the dots. We took care to be kind and succinct because for many of us, we learned our skills from being on the development side of things first.

Vulnerability reports aren’t special anymore. The only ones that felt special were the ones with human touch, the ones doing their job as an adversarial thinker, and taking the care to understand that net positive outcomes require coordination even if both parties don’t see eye to eye.

Nothing has changed. It never was. You’re just inundated with AI slop; which as a practitioner who uses AI regularly I can say with absolute confidence. The end result is the same, the volume is increased, but the special thing was never the report itself.

Finding a vulnerability was always the easy but high toil part. It was the care to communicate succinctly and be invested in the outcome that was special.

Godspeed.

This entire demeanor of the way people and ai people talk about software and technical people is getting absurd and simply unprofessional. It is as if we stopped being adults.

Our solution was to build a tool that uses LLMs to assess the report before it gets to us. Honestly I wish we didn’t have to do this but it works and has really allowed us to spend our time on the actual good reports. (Feel free to check it out at fortworx.com if you want)

> The security researchers are not special, the insight and confidentiality are

vs

> The bottleneck now is not finding potential issues but assessing which ones are real. Unless there’s already a trust relationship, external researchers can’t meaningfully contribute

My take-away from this is that the researchers were special all along and you should probably be building a trust relationship with them.

Despite what I want to believe about tech being a meritocracy, the reality is that trust plays an extremely important role and without it we risk a collapse of our open source software ecosystem.

One of my biggest criticisms of AI is the trust vacuum within which it operates

Right now the rate of signal is high, and the ratio of noise is proportionally high. But it seems like everyone expects the signal to eventually plateau or sharply decline. Almost as if there is a finite supply of "low hanging fruit" for shallow scanning machines to easily discover, and then there will be some kind of new world that follows where only truly difficult problems emerge.

But eventually the question then becomes why even bother with Rust or any other silly borrow checking ideas if we can use more enjoyable programming languages with LLM side-kicks to catch security vulnerabilities on the front side of the development workflow?

IT seems to me if we exhaust all the extant security vulnerabilities to a calculus that asymptotically goes to infinitesimal zeroness, then... the only trick remaining is to scan code before it becomes vulnerable.

This of course made vulnerability researchers seethe worse than aggrieved Redditors.

It turns out he was right all along.

The author also gets it wrong by assuming that regular bug reporters are not "providing a service". They are.

When I wrote up a bug report, I made sure it's thorough with detailed steps to reproduce. It takes a lot of time and I've done it professionally for projects you've absolutely heard of.

Having said that, getting them ignored repeatedly and — even worse — having my detailed PRs rejected, sometimes within minutes, as if I'm some ignorant luser is why I don't do it anymore. My time is more valuable than your hubris.

A lot of open source developers have their heads so far up their own asses they forgot that it takes a community for projects to be successful.

The inbound stuff we get through the vulnerability e-mail is pretty much exclusively spam. Then they start e-mailing random people of the company they can find to get through, and in the end it's still spam.

> Ultimately, it all stems from our responsibility to our users. The security researchers are not special, the insight and confidentiality are, and we need them to keep our users safe. Ignoring a security report communicates you don’t care about users’ security, and it’s rightly a reason for shame.

100%. This was always true and I still think it is. LLMs don't change anything. At most they shift the balance and force a temporary compromise.

> LLMs are as good as almost any security researcher

This statement is extremely dependent on the definition of a security researcher. It might hold if you consider anyone with a HackerOne account, but if you restrict the definition to people who actually put in some effort, it's just not true. LLMs can find some real vulnerabilities, yes, but they also spew unprecedented volumes of garbage that an expert can immediately recognize as such.

> The insight is not scarce and precious anymore. The bottleneck now is not finding potential issues but assessing which ones are real.

Assessing which ones are real should be part of the insight. Real researchers will not submit 150 pages of spam, and three real bugs hidden in 150 pages of spam are not insight. In most cases a researcher will spend significant effort on triage before submitting anything, and an LLM still cannot do that reliably.

> Confidentiality, embargoes, and coordination also don’t matter nearly as much as they used to.

I'd argue these now matter more: the one thing LLMs do seem to do fairly well is figure out specific things based on sufficient information and a scope that's limited enough. So a plain commit containing a security fix is now much easier and cheaper to turn into an exploit than it was before.

> The years of vulnerability reports being special might be over, as weird and uncomfortable as that feels.

I'd hope not. Bug bounties might be over unless someone can figure out the spam problem, but disclosure programs that don't offer monetary incentives are probably just going through a tough period that will eventually calm down as the operators of LLMs realize the costs and do the math.

Unreliable reports have always been an issue and will remain one, LLMs or no. When it gets worse, like in the current influx of LLM-generated reports, the focus should be on identifying reliable researchers, building relationships, and providing guidance on how they can make the reports easier to triage.

Researchers are not special, but the insight they can provide totally is. LLMs might force everyone to make better use of that insight, instead of just consuming bug reports and drowning in triage.

[1] https://groups.google.com/g/golang-announce/search?q=juho

I filed a report, they marked it as 'informative' and thanked me, recommended I keep looking for more vulnerabilities, but no payment at all; they said I had to be able to demonstrate major disruption of service... Which I presume is illegal. I literally showed them all the ingredients of the attack, the exact curl commands, payloads, the exact response delay could be easily be verified; you could see the server response slowing down proportional to the degree of nesting in the payload. I could execute it without authentication too; so it was essentially certain that the attack could be scaled but they made it impossible to get a reward.

The hardest part was writing the report which took several hours.

So yeah, 30 minutes of looking for a vulnerability, no prior experience in security research, first project I looked into on Hacker One, ever... A company in crypto sector which is a major target of hackers and takes security relatively seriously.

Imagine how insecure most software is! Imagine how bad most vibe-coded software is especially! Companies might as well run their servers directly inside Kim Jong Un's data center in North Korea.

North Korean hackers probably have a dashboard which shows more detailed and accurate platform analytics than what the founders of the company can see.

It’s tough staying motivated on a craft when an AI is nearly as good as you. Chess players manage to do it at least.

No they are not. Everything else can be safely ignored. The author is suffering from AI psychosis and needs to get some help.

Oh really? If LLMs were as good as almost any security researcher then you wouldn't be getting flooded by bullshit reports from them. You'd be receiving legitimate reports instead.

I don't think the gift analogy works well. In most cultures, turning down or even ignoring a gift is considered anywhere from impolite to hugely offensive. But that's the opposite of open source: there's nothing wrong with requesting changes to a PR or even closing it.

Is this even a question? You triage and fix the vulnerability just like any other one. Are truths spoken by folks one dislikes — even for perfectly valid reasons — any less true?

The only way I can imagine this somehow applying is if someone has a habit of reporting vulnerabilities which do not exist, or of exaggerating their severity. Is crying wolf a CoC violation? If so, then I can imagine that particular sort of bad behaviour justifying some consideration before acting on a report.

My sweet summer child, the "profession" will become an agent managers talk directly to, like many other professions.

What is this, rage bait? It's bullshit, and insulting to actual security researchers.

That might be true for low-effort vulnerabilities and fake security researchers, but the real security researchers are far from being replaced by LLMs.