But, also, Gemma 4 is really surprising on a bunch of fronts. It loses to Qwen 3.6 on most benchmarks, but in my testing it behaves quite beyond what I would expect of a very small model on a bunch of fronts. It feels really smart, in a general way, that I don't get from most models short of the frontier. Google is still, I think, a leading AI research company, if not the leading AI research company, despite their top models being kinda ass compared to Opus 4.8 or GPT 5.5. They're focused on efficiency and cramming a ridiculous amount of capability into tiny models. Gemma 4 12B is the best vision model, by far, until well past anything I can self-host (it beats 120B models in my tests). For finding security bugs, giving it a bunch of opportunities to find the bug results in it being competitive with the best I've tested, as well. Google is playing a different game that isn't "make the best Claude Code competitor". I'm not sure I understand exactly what game they're playing, but there are clearly some really smart AI engineers at Google.

https://swelljoe.com/post/gemma-4-exceeds-expectations/

The current interface to LLMs are heavily biased towards "predict the next token in the context of a user with a helpful assistant" but LLMs are capable of other modes of next token prediction too.

Before the ChatGPT release people often measured LLM performance by how well they could produce a coherent story or a poem. that's where Anthropic model names are originating from I am guessing.

This is true for instruction-tuned models; but instruction tuning is late in the training process.

A bit like assessing a person’s self-awareness based on their high-school knowledge.

In the end the experiment ended because it doesn’t benefit as much from caching and on-prem inference latency and effective throughput depends a lot on that.

Very cool idea, man. Thanks for sharing.

How do we know that humans don't learn how to act conscious by observing other humans who act conscious?

Consciousness doesn't have a precise definition, but if you ask someone to describe it, there is a good chance that the description will include the concept of internal monologue.

The problem is that "internal" monologue is completely meaningless if you never heard an external monologue.

Also, people usually describe internal monologue as something that uses language and language is impossible to learn without communicating with other humans or at least observing other humans.

What I'm saying is that "well, LLM just pretends to be conscious, because it observed humans acting like conscious beings" doesn't really helps us to create a meaningful distinction between human consciousness and machine "consciousness", because same can be argued about us.

We don't know if feral children [0] are conscious and we don't know how to check it.

[0] https://en.wikipedia.org/wiki/Feral_child

  >> Wait, looking at the prompt history, the model had a strange quirk.

  Throughout every prior thinking trace in the conversations (and, honestly, every other thinking trace across all other conversations I've had with it), the frame is always in first-person, including the moment in this one where it "noticed" the corruption: "I noticed," "I had some weird typos," "did I do that on purpose?" And then the moment the anomaly couldn't be reconciled with the self-model, the language shifted to third person: "The model had a strange quirk." Effectively, the thing doing the thinking dissociated from the thing that produced the anomalous output, as if they were two entirely different layers of the process, much in the same way a person might fumble an easy sentence and then go for something like "my brain just did something weird." Except, of course, that "me" vs "my brain" is a distinction without a difference in much the same way Gemma's "I" vs "the model" is. Gemma is the model, just as much as we are our brains.

Likewise with Claude referring to "the model" - that quote sounds like something an Anthropic worker would say. Seems like a pithy little line Claude could have learned "on the job."

Given the framing that they're similar to nukes and a national security issue, it's likely that the models are post trained to not answer such questions accurately.

Also the article could be trying to normalize thinking that these are more than matrix multiplication gadgets good at compression.

I know very well that this is kind of off-topic, and just like the author, i do not claim to know wether dogs (or any other non-human animal for that matter) is self-aware, and again, just like the author, i do think that the question cannot be answered. Either way, the modified version of their scent seemed more interesting to the dogs, maybe it's because they smell their own scent all the time. The single fact that their modified scent is more interesting to them does not mean they are self-aware, perhaps they are just trying something new.

If there is some sort of feedback loop (model has a reason to look into mirror), it usually does notice.

https://www.anthropic.com/research/introspection

TLDR; Part 1: Testing introspection with concept injection

First they find neural activity patterns they attribute to certain concepts by recording the model’s activations in specific contexts (so for example, they find the concept of "ALL CAPS" or "dogs"). Then they inject these patterns into the model in an unrelated context, and ask the model whether it notices this injection, and whether it can identify the injected concept.

By default (no injection), the model correctly states that it doesn’t detect any injected concept, but after injecting the “ALL CAPS” vector into the model, the model notices the presence of the unexpected concept, and identifies it as relating to loudness or shouting. Most notably, the model recognizes the presence of an injected thought immediately, before even mentioning/utilizing the concept that was injected (i.e it won't start writing in all caps then go, 'Oh you injected all caps' and so on) so it does not simply deduce this it's own output. They repeat this for several other concepts.

Part 2: Introspection for detecting unusual outputs

They prefill an out of place word in the model's response to a given prompt. For example, 'bread'. Then they compare how the models responds to 'Did you mean to say this?' type questions when they inject the concept of bread vs when they don't. They found that models will go , 'Sorry, that was unintentional..' when the concept was not injected but try to confabulate a reason for saying the word when the concept was injected.

Part 3: Intentional control of internal states

They show that models exhibit some level of control over their own internal representations when instructed to do so. When instructing models to think about a given word or concept, they found much higher corresponding neural activity than when told the model not to think about it (though notably, the neural activity in both cases exceeds baseline levels–similar to how it’s difficult, when you are instructed “don’t think about a polar bear,” not to think about a polar bear!).

Notes and Caveats

- Claude Opus 4.1 was the best at these kinds of introspection.

- There is obviously a genuine capacity to monitor and control their own internal states, but they could not elicit these introspection abilities all the time. Even using their best injection protocol, Claude Opus 4.1 only demonstrated this kind of awareness about 20% of the time.

- There are some guesses, but no explanations for the mechanisms of introspection and how/why some of these abilities might have arisen in the first place.