“Prompt injection attacks have become, to agentic AI, what SQL injections were to web applications: a systematic, category-wide vulnerability class that requires the same systematic strategies and defenses.”

Isn’t prompt injection far more fatal to LLMs than SQL injection is to SQL databases?

Like, the problem of SQL injection was that user input was forming part of the instruction string given to the SQL engine, and so malicious user input could include various SQL grammar terminals to end the current SQL command, followed by complete SQL commands of their own, and the engine would simply execute both commands. The fix was prepared statements: fixed/static/pre-compiled instruction strings, that can only ever perform fixed/static/pre-defined logic, and that logic can then be (more) safely applied to arbitrary user-input data.

The analogous mitigation for agents is to have fixed behaviors they can perform, such as “read repo 1” “read repo 2”, etc., and the user input is used as data to select which of these fixed behaviors to execute. But we already have this technology - it’s called a menu. The value of LLMs is specifically and intrinsically predicated on being more than a menu, while the value of SQL does not depend on being more than “pre-set logic operating on arbitrary data” - user input being part of the instruction string to SQL was incidental, for developer convenience.

This is like setting up a normal CI job with access to secrets and running it on public PRs. If you configure GitHub to allow public code or LLM instructions to run in contexts that have access to sensitive things, they will leak; that’s not GitHub’s fault, it’s yours.

an agent responding to public issue should only ever see context limited to that repo.

it seems like with the evolution of AI - we are slowly missing out basic security practices.

Or was it the agent itself that somehow had elevated permissions? If that's the case, you've misconfigured the agent... we know that agents cannot be trusted to enforce anything.

Why does this section not have when it was fixed or GitHub acknowledge/rejected this?

Did they not fix this?

The difference is that (A) SQL is deterministic and (B) SQL implements internal access control (and how well that works).

Prompts from non-authenticated user should have no access to any private repositories. The real question is: can you trust MSFT GitHub with your code, now that “outsourced” engineers are supporting it?

If you don’t want an AI Agent to read private repos then you do not give the AI agent access to the private repos. This is not a permission bypass issue but a prompt injection issue which can’t be reliably solved at the Agent layer

2. Or issue is not solved yet by GitHub, and meanwhile bad actors gonna try vulnerability on repos. Due to number of repos there is non-zero probability. But as with scams almost nobody’s going to admit the leakage.

Anything else?

* Trigger the workflow on issues.assigned events in GitHub * Read the issue Title and Body * Post a comment in response using the add-comment tool * Run with read access to other repositories (public and private) in the organization "

Self inflicted damage, I think. So what is their claim, that gh-aw's "Safe output gate" and "Threat detection" didn't stop the workflow?

We deliberately gave GitHub's AI Agent permission to access both public and private repos and then tricked our configured agent into leaking private repos.

It's almost indistinguishable for me as a single user working on a codebase and I get no AI, no multinational corporation looking at my repo, I have complete control and will never be locked out of 'my' account because some company decided to do it to me.

What is going on over there? No process, no oversight, just YOLO? Super-scary, because it means other stuff that we don't see is likely to be done in a similar manner.

It can't reliably learn so stop trying to teach it. Lock the bedroom instead.

Look I like interacting with these tools as much as the next guy, but I'm certainly not going to trust them with access to information and then allow anyone to send them prompts.

Edit/further thoughts: So (assumable as they said this is disclosed with github's knowledge) this has been patched. But how many different word combinations will it take to find another way to have this occur?

1) Sandbox any LLM that has access to tools (I don't mean the pathetic sandboxes the agent harnesses provide).

2) Assign them credentials and use auth/access control like you would for a human.

How on earth is a probabilistic token predictor supposed to turn untrusted user input into trusted system-level directives? The strict trust boundary must be maintained on this side of the agent, not within it.