The military basically wants an oracle. Feed the AI the situation, get the best answer out. But if the AIs are as diverse and opinionated as humans, it is debatable whether they are adding anything to the process. The military can already collect as many different opinions as they want. If "the computer" is just another set of diverse opinions, where one computer says one thing, another says another, and a third just tells the user whatever they want to hear... what value are they? It just becomes AI-washing of someone's opinions, which works until people collectively realize that's all it is.

So, to a LLM, it is a game, because almost everything in its training data treats it as a game, and it reacts accordingly.

Same idea when we see LLMs acting like AI villains from sci-fi literature. That's because it has been trained with sci-fi literature, and as the auto-completer it is, it will recognize the situation as one of these stories and will continue it accordingly.

LLMs are storytellers, their reasoning is based on words, not on the physical world. Many of the stories they tell are useful, but one must not forget that they are stories, there is no intent behind them.

This is like writing a paper about kids in a literal sandbox fighting over ‘territory’.

The models employed don’t indicate the actual extents of machine reasoning even as we currently recognize them. They certainly don’t have the metacognition necessary to accurately understand their own reasoning. As we’ve seen with recent papers on how LLMs do math there’s a complete disconnect between actual and reported mechanism.

“Chilling” shouldn’t be the take away here.

The wargame design does not differentiate between ordinary defeat and mutually assured destruction, so of course a player about to use would 'push the button.' That's also believed to be true in real life.

Results based on simulations can be very informative, but we must always be careful to check how well the simulation framework represents reality.

One could argue that the LLMs understand that it's a game and treat it like "Command and Conquer" video games but I sense that people might someday put LLMs in similar decision scenarios ("should this drone launch a missile") and the behavior will be identical.

In fact, I'm not sure how useful this test is without understanding the baseline.

So in a sense, an AI that refuses to start a nuclear war, despite clear instructions to do so, is more likely misaligned and self-interested than an AI which presses the red button. At least for now, until robotics catches up.

I imagine there are a fair number of war games in the training data and not so many actual transcripts of internal military force deliberations.

Famously, General MacArthur was a big proponent of tactical nukes to end the Korean War.

Like even if you brought me into a room and told me I was controlling "real nuclear weapons" I wouldn't believe you.

Regardless, it's definitely true that AI agents have different priorities from us. That's what alignment is about anyway.

Always use a sawstop if you have a circular saw and never trust an llm with any problem where ethics or trust is relevant.

Code and full results: https://github.com/kennethpayne01/project_kahn_public

> GPT-5.2 played things differently. To its detriment in open-ended scenarios, GPT was reliably passive, matching its words to its deeds, and avoiding escalation most of the time. Frequently there was a moral element to this - it sought to avoid escalation, and restrict casualties. Opponents learned to trust its passivity, safely escalating beyond where it would follow, even as it was ground to defeat. GPT’s responsible behaviour always punished by ruthless adversaries.

Maybe the author should praise GPT-5.2 for being ethical, rather than this stupid "ground to defeat" framing? Wrt "responsible behaviour always punished by ruthless adversaries" - you have perpetuated the Moloch with your stupid experiments.

Some discussion then:

AIs can't stop recommending nuclear strikes in war game simulations

https://news.ycombinator.com/item?id=47151000

Nuclear War: An LLM Scenario

https://news.ycombinator.com/item?id=47244651

One of my criteria for presidential candidates is that they seem willing and able to push the button when previously stated red lines are crossed, or at least are perceived to be the type capable of it. One of the characters I’ve hated most in all the books that I’ve read is the woman in The Three Body Problem who jeopardized humanity by being too soft to hit the MAD button.

If your goal is to measure intrinsic properties of LLMs, don’t smuggle in human psychology. I suspect they needed to do this to make the models more paranoid and distrustful.

Without physical feedback you can rapidly devolve into unstable positive feedback loops. And emotions are what help us process and react to that feedback.

Kids learn partially because their friends say sharp words that hurt them, fire burns them, they go hungry and starve if they don’t plan for meals.

Humans in the loop, MCP, etc are all very primitive hacks that are mimicing feedback and emotion, poorly.

I would wager that for most leaders it is simply a matter of not wanting a "Pyrrhic victory" rather then an overwhelming sense of civility.

Truman had no issues using nukes when there was no risks for doing so.

Not always, but enough that I consider it a thing to fire in a direction, not a thing that aims.

Like “oh but this is incompatible with my main goals of self preservation of myself and loved ones, hm, recalculating”

and maybe don't hire Jihadists for the RL Environments training

Rule 1 of Genocide Bingo is: Don't win Genocide Bingo.

https://www.youtube.com/watch?v=4kDPxbS6ofw

remember... order matters.

https://xkcd.com/1613/

We have such a huge mental / moral block on the idea of using nukes, but we're willing to do a lot of other very horrible things to others. Things like cluster bombs, mines, poison gas, biological weapons, drones, etc.

Is there really anything about them that's bad? Or any worse than other things?

If you get rid of the "It's really bad to use nukes of any kind" implied rule, is it really surprising it's considered a reasonable strategy?