- You shall not embed copyrighted material in your models.

- You shall not bombard every little website in existence with 1 million scraping queries per day.

- You shall not use your political influence to pump and dump your AI (or rocket?) company.

- You shall not imperill the whole IT sector by buying all CPU and memory chips.

These new rules will affect every society directly in a positive way. Thanks.

> AI has become a major commercial technology

>Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety

> AI companies that develop advanced AI models must have strong security standards that protect their model weights

Anyway Dario's financial interests aside. This is an interesting breakpoint for me.

> Second, any response to AI-driven job displacement needs to address both the need to provide for everyone economically, and the need for people to find meaning, purpose, and agency. The latter is ultimately more important

To me this reads as an out of touch statement. I think the majority of people on earth work to keep a roof over their heads. Of course work can be a source of meaning, purpose, and agency, but to call it the more important aspect on a societal level is a sort of rich person like Dario statement to make.

So, basically, make open-weight models illegal. It's nice for Dario to come out and say this so explicitly.

People get income from one of three places: capital income, labor income, or the welfare state. If this technology truly unlocks a holy panacea of productivity with a commensurate drop in employment then capital’s share of the national income can and should provide for a wider and deeper welfare state. Nothing new need be invented here. Dario’s long and only somewhat organized list of policy interventions makes appropriate preparedness sound like a manic pulling of any and all levers when a simple theory of distribution will suffice.

I agree on some points about the missuse of AI particularly for surveillance, military and propaganda.

But this reads like a post further glazing Mythos, and we are just one or two years away "trust us guys", and similar to Mistral's policy plea "please use AI everywhere or we are going to be left behind".

I had the hardest time accepting one of his first points that LLMs could barely write a line of code 4 years ago.

ChatGPT 3.5 was reasonable at code writing but hallucinated a lot of library functions. Yes we have better harnessing today, and models have been further finetunned with reallife code, but pushing this argument just to support his exponential narrative is deceptive. Like most AI marketing.

Okay, I don't understand how legitimate access is granted then. Surely, Dario isn't saying to ban Sonnet, because I can definitely make it do cyber harm, as most exploits that I've seen in the wild with my own eyes were trivial.

So the only way I see his proposal working is:

- No open weights, AI is centralized in the hands of few

- We get AI-FAA that sets the rules and monitors

- If I want to do a security scan of my codebase, I get a time and scope limited license from AI-FAA that I upload to claude that will allow it to run the security scan in cloud with their models - Claude Mythos Scanner(TM).

Dario's proposal ultimately requires that people lose direct access to inference via API. Is this why they've been building SaaS clones with AI bolted on?

They are asking for FAA style preclearance and third party audits. That literally means no new AI startup can emerge. Do they not know that audits cost money?

Protect your own monopoly, protect your customers' regulations. They want strong regulation like the FAA to raise barriers to entry for the foundation models they themselves build, but then why do they want to loosen FDA regulations? While at the same time driving token consumption from their own customers.

They talk about permanent job displacement and UBI. I usually call this "a morally packaged safe landing."

They are doing something unpopular (destroying jobs) and getting criticized for it. But they do not want to be criticized further, and they want to ask for social sympathy. So they claim a 'noble cause' that everyone can sympathize with and that is safe for themselves

AI will generate astronomical productivity gains and capital profits, which AI companies privatize. So why should the social costs be paid by national taxes? In my opinion, something like "We will donate all of our AI companies' revenue for the next 10 years to society" would show genuine sincerity.

Then they say, if we do not develop AI, China will eat our lunch, and they go after China. But is not this really about preventing Chinese dumping, maintaining our own token prices, and asking the world to beat down China so that they can preserve global tech hegemony?

But by blocking China from the CUDA ecosystem, now the CANN ecosystem has emerged, has it not? If China develops techniques that reliably reduce inference costs, who knows how things will turn out then.

Honestly, I like Anthrpic's Claude, but the Anthropic CEO's rhetoric is so stale. It is not that it feels hypocritical. It is that this is just a one dimensional rhetorical tactic that assumes the public is stupid.

I do not think open source is unconditionally good. (It is good, but it can become bad in all situations or all countrie). Open source itself is a barrier for countries outside the Anglosphere when they want to release IT products. Because there is no incentive to buy a product that is worse than an open source alternative. So I do not think everything necessarily has to be open source.

But this (referring to Anthropic's position) seems to treat people like fools. If regulation is needed, shouldn't they also argue that FDA regulation is needed? I wish they would be consistent

If that arrogance was well placed at least you could somewhat excuse it, but the fact that it is so overtly hypocritical and based on false premises just makes it so much worse.

I feel significantly less sympathy for Anthropic's Supply Chain Risk designation if they believe the government should have this power over them. You get what you sign up for.

- And this were the first steps of Anthropic establishing worldwide corporate technocracy.

- And this is when Anthropic lost and everyone got access to AI.

Similar to how IBM's defeat allowed us to have PCs.

How much of the policy prescription changes if the exponential is actually just a series of sigmoids[1]?

[1]: https://x.com/ylecun/status/1799064075487572133

This is a somewhat ironic take from someone who very publicly feuded with the US government about whether their AI could be used for waging war.

Without direct workforce or policymaker representation on the boards of private entities, the private sector will seek to maximize shareholder value even if that means workforce reductions.

It's not clear that any country could realistically ensure that incredibly powerful industries/private sector entities operate perfectly aligned with national interests, short of nationalization.

Large tech companies are already quasi-state actors. In theory, international law and regulations can be binding and enforceable. We see how well that works in practice.

I understand why Dario thinks this is crucial, but it's a very dystopian view of the medium-term future.

I'm not an optimist to the point that I believe that AI will lead to global Star Trek-style utopia (although it theoretically could), but ongoing disparity between "allied" and "enemy" powers relating to hardware technology and software models is both not really possible to enforce in the long term, and a pretty dismal state of global affairs even if successful.

I'd be interested in an expert geopolitical opinion on what the long tail of this would really look like in any sort of reasonable reality.

"Don't mention the (Iran) War!"

This is a massive exaggeration. The advancement in the automation of computer code writing has been impressive and is obviously, at least in the short term, changing the software engineering industry substantially. Most other fields have not been affected to nearly the same degree. Certainly not biology, physics, finance, and law (I don't know enough about the math and translation fields to speak to those).

---

"3. Accelerating AI’s positive impact..."

This whole section is the type of thing that often comes out of the mouths of Silicon Valley tech executives without a pharma background. It indicates a thorough lack of understanding of the realities of pharmaceutical research. What he is describing here is removing many of the solid, evidentiary rules that are in place to make sure that the drugs reaching the market actually work and replacing them with proxy predictions. Look, my least favourite part of the job is the animal testing, and I would be hugely grateful if that could be eliminated from the drug discovery pipeline. People have been trying to do that for a long time. But it's extremely difficult. Biology is very, very complicated. Our understanding of how processes in organisms work are vague and approximative. This is not computer code. Even if Anthropic somehow got all of Big Pharma to hand them their proprietary data, it would only scratch the surface of the understanding that is needed to solve these kind of problems. Due to these realities, the program Amodei is describing here would, effectively, open a floodgate of drugs on the market that don't actually do what they are supposed to and are more likely to have unidentified toxicity.

-C.S. Lewis

Gasoline has gone from barely being able to power stationary farm machines to now being the fuel that underpins our entire economy. So, great news all around, right?

> which predict an exponential increase

And was that actually delivered?

Real question: If a model goes from 80% accurate to 85% accurate is that an exponential increase in "cognitive capabilities?" Are we considering training costs and effort?

What makes me doubt that Dario Amodei has really internalized the problem is the lack of humility, the stance that it's just important that the "good guys" keep the technology away from the "bad guys".

If you really want to provide AI with public benefit, you need to prevent power concentration. How? Some unpolished ideas, I'd be happy to hear yours:

- Avoid getting too close to an administration that is openly attacking democracy and is not interested in the benefit of humanity or mutually beneficial cooperation.

- Don't support surveillance. Non-(US-)Americans have human rights and privacy, too. Prepare for a situation where a government tries to convert your compute infrastructure into surveillance infrastructure.

- Support the creation of community data centers. In other words, build data centers together with local communities and make sure they profit from them.

- Advocate for laws that require transparency about resource usage and emissions of data centers.

- If you don't want an AI race, make sure that other countries don't need to fear the US concentrating too much power. Create institutions that can be trusted by other countries, too.

EDIT: I forgot:

- If qualified labor will actually turn out to get devalued, we also need a plan to prevent states from turning into rentier states that don't depend on a well-educated society any longer.

I like to stay up to date on things but more and more I’m finding myself pointing codex at a URL and saying “get to the point”.

I'm not going to claim that the CEO of pre-IPO company has no incentive to bolster the claims of his tech, but to completely disregard everything he is saying based on that seems awfully binary.

I don't know whether people are just high on copium, spouting "it's just fancy autocomplete" or "only humans can really be creative" on every LLM-related thread, but it is impossible to deny that in a span of a few years we've gone from models that could barely put together a sentence, to something maybe not equivalent to a junior developer, but at least resembling it.

And sure, you can point out every flaw that current day LLMs have, just how everyone pointed out that Stable Diffusion couldn't generate accurate hands (until it could 6 months later!). But the gradient is pretty clear and I am yet to see a well-argued narrative from anyone why scaling laws should fail in the next year or two (by which point it feels like we're going to have a real problem, extrapolating the current trajectory).

I'm very glad this discussion is at least being had, and I wish everyone would get off their high-horse and take things a bit more seriously.

People are not in charge, and have not been for some time.

Corporations and governments select their leaders and policies to advance their interests. People fit the work, not vice-versa. Only external competition or internal capabilities limit them (i.e., predation or resources).

External resources have been optimized as profit and exported costs; now that AI replaces the pesky need to source elites, capabilities will expand, which will result in more competition.

Law is not a lot more than settled expectations, and increasing capabilities changes expectations much more than even market disruption and disinformation, so I wouldn't expect law to save us.

As far as I can see, only competition will temper things, and only if AI companies are seen as responsible for their customers' applications - which I doubt. I say this in hope of being proven wrong.

As annoying as their tone is, the real big danger is what they are setting up for. All this fear-mongering around Mythos, the overly aggressive controls on Fable, and these manifestos they keep writing, are part of setting up for REGULATORY CAPTURE. Even collaborating with the Pope and the Interfaith Alliance (https://iafsc.org/our-work/faith-ai-covenant) are part of creating a vast support network for regulations and restrictions. Those regulations will help those faith organizations or the government or whatever, but will also help Anthropic’s bottom line.

Those regulations will not support your civil liberties. They will restrict speech, access to AI, and allowed uses of AI. They will lead to bans on use of models from some countries like China, and also bans on open-source or open-weight models.

If Dario wants to be trusted, he needs to explicitly say in writing that Anthropic will not support any legal or regulatory restrictions on open-source AI, open-weight AI, or Chinese models. Otherwise, what he is really saying - even as he claims he is trying to ‘defend democracy’ - is that he and Anthropic do not truly support fundamental rights like our right to speech.

It’s not just Anthropic either. OpenAI had their own recent polemic, pushing for regulations like mandatory safety reviews by agencies for “frontier” models (https://news.ycombinator.com/item?id=48387246). It’s a dead giveaway that these companies have no moats, are in serious danger of being a commodity, and are now in the process of using regulations and enshittification to hold onto money and power.

A good proposal here is: should Anthropic and OpenAI become sort of VC's that fund other competitors?

We all want to nuclear codes so badly. We are addicted to intelligence and labour so badly that we simply can't concieve that a pro-social actor might want us all not to have it, and for good reason.

I mean... Obviously, insiders like Oppenheimer (who dedicated their lives to considering the implications of the technology under discussion), they just feared nuclear proliferation because they wanted all the profits for themselves, right :(

It's not clear to me on which side of the coalition USA is meant to be in this divide. And as an European I'm not sure whether being in China's or USA's coalition is better in the long term.

In general, this deliberate mongering of ever more geopolitical division is extremely harmful. As is the Trump bootlicking.