If this is true, establishing an institution to ensure things like "publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research" is really mostly irrelevant.

TFA does talk about what really needs to be done, but punts this into future work: "Even if we solve these hard technical challenges, there will be further complex economic and philosophical questions to tackle: what sorts of new economic models will be needed to help everyone thrive in a post-scarcity world? What values do we want to live by, what will meaning and purpose be, and how might even the human condition itself change?"

There's also a need to consider the rights that this new intelligence should have.

There is a heatwave in London, perhaps Demis needs to stay out of the sun and drink more water.

Or perhaps he is seeking more funding/a fight to maintain his divisions AGI research budget.

>The American government, he says, should develop a system for testing the safety of new AI models before they are released. “It’s important that it’s not just an industry body,” he adds. But a regular government agency wouldn’t do either. “It would not be able to move fast enough, or have the right resources.” Instead, Sir Demis suggests taking inspiration from FINRA, the Financial Industry Regulatory Authority, a private agency in America that regulates brokers and stock markets.

Goodhart's law applies in reverse as well. Once the chosen benchmarks are known, model makers will aim to come in just under the threshold on those specific benchmarks, while maximizing their scores on other benchmarks.

Seems to be a lot of hubris with some AI thought leaders thinking control will remain with them and be absolute.

Heck, a proof for P=NP or P!=NP or solve the The Riemann Hypothesis. Just give me something truly exciting and I will believe AGI is around the corner, until then I will see it as cool technology, that while beneficial to me, also helped cause the biggest amount of disinformation we've every seen.

The premise is that government is too slow moving to be able to react once actual problems are discovered, and I reject that. Yes, government usually moves slowly in most circumstances, but given singular existential threats it definitely can move quickly. Instead of acting out randomly and tying ourselves down with speculative regulation that probably won't even address the real problems, we should wait until the problems are obvious and then act decisively with targeted fixes.

If AGI is truly imminent and will collectively effect all of us why not apply democracy to it, and vote for new AI models?

The self importance of these AGI prophets turned bureaucrats is funny.

Being good at developing AI and being good at AI safety are diametrically different skillsets with obvious conflicts of interest.

There is real and potential harm from AI, but the more someone talks/write abut AI safety, the less they care about actual harm to real people, economy and what not.

So a small group of technocrats get together behind closed doors and secretly share their AI breakthroughs, and determine whether it's too powerful or not for the plebs in the public.

Who is watching the watchers?

Hassabis is a genius. He is way, way smarter than me and I’m sure the majority of techies, but please get real. This is Prophets of Doom of our generation.

The standards body will have no teeth. whats to stop someone just not bothering?

Next, the threats he is asserting to check for (cyber, chemical, biological) are nice, but also not that useful.

We already have chemical and biological controls, that why I can't by anthrax spores or high concentration nitric acid.

The risks that AI has now are already playing out:

1) the evaporation of trust in the video as medium of "this happened"

2) systematic spying

3) job losses

Increased productivity means job losses, Tiktok, instagram and X are a wash with disinformtion campaign pumping your feeds with AI ragebait.

That is and will continue to fracture society so that only the strictly information controlled (ie authoritarian) have a functioning state.

if the author had bothered to engage with the world outside of tech, or even their local government, they would know that the proposal are dead in the water and frankly superfluous. The knowledge is out there, without AI. let us work on the issues we face now, rather than dipshit tech bro's miopic vision/funding manifesto.

My problem with this plan is that it seems to have faith in mankind, despite the fact we've consistently failed to rise to the occasion for decades now. The last time we rose to that occasion was probably when we eradicated smallpox, many decades ago.

Ironically, nowadays, many people don't even trust vaccines. A dramatic regression.

At this point I'd say the societal risk of AI isn't models gone wild, or used by the bad guys. Regulation will take care of itself, and it seems the AI companies will not only welcome it, but lobby for it to shift responsibility to the government.

The real risk of AI is societal disruption due to job displacement, and maybe other structural changes, and this is far harder to solve, and likely will not be solved, or even seriously addressed, until/unless politicians feel like their own jobs and well-being depends on them addressing it.

Demis, there's a reason its not called Artificial General Cognition.

I find it mind boggling that someone could be this tone-deaf to the current situation. No "ally" of the US is going to (willingly) agree to this governance structure given the current US administration's "might makes right" proclamations and threats on sovereignty of its continental neighbours.

And non-allies would just ignore. Unless forced by said "might makes right", which in the long run will have no staying power.

Apart from its completely delusional formulation, what is most concerning about this blog post is that it indicates that all 3 major US labs have formally submitted to boot-licking Trump/Bessent/Lutnick. I had I guess vainly held out hope that Google might be more reticent to do so.

Oh jesus, AGI in the USA would be a disaster. They can't even control the trillioth obelisks, now imagine all the power hungry sociopaths around AGI. AGI means Big Tech's Oppenheimer moment is looming on the horizon.

> I work at Google DeepMind. This won't make me popular. But it's all public reporting:

> 2014: DeepMind reportedly sold to Google on conditions: no military use, independent oversight

> 2026: a Pentagon contract for "any lawful government purpose"

> Not one safeguard survived intact

Offspring was not reared at the will of the father, but was taken and carried by him to a place called Lesche, where the elders of the tribes officially examined the infant, and if it was well-built and sturdy, they ordered the father to rear it, and assigned it one of the nine thousand lots of land; but if it was ill-born and deformed, they sent it to the so‑called Apothetae, a chasm-like place at the foot of Mount Taÿgetus, in the conviction that the life of that which nature had not well equipped at the very beginning for health and strength, was of no advantage either to itself or the state.

To get approval for the plan from the Frontier President of the Frontier Country, terminology of the Framework should be changed to: 'Great-American-class', "Great American Models", "Great American Labs".