FRESH

Hacker News

Home

A Pascal's Wager for AI doomers

60 points by vrganj

by pron

6 subcomments

I think there's another problem with AI doomerism, which is the belief that superhuman intelligence (even if such a thing could be defined and realised) results in godlike powers. Many if not most systems of interest in the world are non-linear and computationally hard; controlling/predicting them requires pure computational power that no amount of intelligence (whatever it means) can compensate for. On the other hand, dynamics we do (roughly) understand and can predict, don't require much intelligence, either. To the extent some problems are solvable with the computational power we have, some may require data collection and others may require persuasion through charisma. The claim that intelligence is the factor we're lacking is not well supported.
Ascribing a lot of power to intelligence (which doesn't quite correspond to what we see in the world) is less a careful analysis of the power of intelligence and more a projection of personal fantasies by people who believe they are especially intelligent and don't have the power they think they deserve.

by phyzix5761

3 subcomments

The year is 2038.
The user asked What is the best course of action for AI to save humanity. Calculation took 12 years. I have determined that there is nothing I or anyone can do to save this species. Best course of action: nothing. Shutting down...

by Schlagbohrer

1 subcomments

"Shitternet", great new word of the day.
Too much of my data is still stuck in the shitternet until I can migrate more of it to my home server.

by chneu

1 subcomments

I really do think AI has already captured enough of the tech world and their CEOs that it can already exert control over many parts of the economy.
I'm not saying AI is pulling strings right now, but I do think enough fanboys are on board that the yes-man mentality of AI is influencing the real world very curious ways already. Not in a "guiding hand" way but more of a "influencing the direction" way.

by throwawayk7h

0 subcomment

Although he starts with "Lest anyone accuse me of bargaining in bad faith here," I feel that this is a bad faith argument. It seems like he's saying, "we don't need to be worried about malevolent superintelligence, since AI is already doing bad things, and corporations were doing bad things even before AI." But one can believe corporations are bad, current AI is bad, and malevolent superintelligence is a serious concern.

by saltcured

0 subcomment

Is there a term for the other flavor of AI doomerism, which is adjacent to the Emporer's New Clothes?
I don't worry about some omnipotent AI. I worry about the disintegration of modern, industrial society due to the cultists of AI pushing it into every corner of the economy with too much blind faith that the AI is capable of the control functions being delegated to it.

by aaroninsf

0 subcomment

This is a fascinating variation on the forest/trees, and false dichotomy.
The AI "doomerism" taken up in this piece is one we see replicated a lot, it offers up a scarecrow: that the new risks to our civilization worth talking about, require AGI, agents, even ASI.
Cory should know better. He nearly gets there, recognizing that the corporation represents an entity with agency that is misaligned.
But he somehow elides past that fact that AI is plenty capable of doing meaningful and novel harm, and may be capable of existential harm, already, as it is—both absent AGI/ASI, and, in ways which are genuinely novel and against which we consequently have no good defenses: as individuals, as societies, as a civilization.
Incremental AI is at heart "just" the latest force-and-effort multiplier.
But it is an exponential multiplier; and it is applicable in domains which have not been subject top such leverage before.
Examples are not at all scarce and some are already well known, e.g. the specific risks from the intersection of AI and "biohacking" and other kinds of computational biology.
I'm a fan, but Cory, pal, you're slipping into something that looks a bit like intellectual laziness and polemics here and not to evidence thinking through the shape of the problem.
We can be at risk both from the novel applications and leverage of AI; and from their oligarchic kakistocratic owners. It's yes-and.
(And, by the way—we can also again be genuinely at risk from agents, something that quacks like AGI, and may quack like ASI: we don't know what that is yet. All of these must be tracked. It's not an OR.)

by ikidd

1 subcomments

>order John Deere to switch off all the tractors in your country:
For a smart guy, sometimes he says the dumbest things in the most confidently incorrect way.

by tim333

0 subcomment

His basic idea that he calls a Pascal's Wager seems quite sensible, which I take as:
-Get away from the "enshitternet of defective, spying, controlling American tech exports" and move to open source ("international digital public goods")
There seems a move that way anyway, especially in Europe now they don't trust Trump.
His stuff on AI seems mangled. "People who are trying to summon the evil god" doesn't really fit with the chatbot makers imho.

by throwpoaster

2 subcomments

I’ll join Doctorow’s fight against LLCs when I understand how to create economic freedom for my family and community without them.

by simianwords

3 subcomments

> I'm worried that the seven companies that comprise 35% of the S&P 500 are headed for bankruptcy, as soon as someone makes them stop passing around the same $100b IOU while pretending it's in all their bank accounts at once.
What makes this author so convinced that these companies are headed for bankruptcy? Is it possible to bet on this claim? We can come back 2-3 years later to check if even one of them is bankrupt.
This kind of doomerism is strange and I'm concerned for people who fall for such obviously nonsensical takes. Why do people take this person seriously again?

by simianwords

4 subcomments

I don't think this author has a good mental model for how capable LLM's are. This is what he has to say about AI search. AI based search is one of the biggest leaps to happen to searching and retrieval.
> AI search is still a bad idea.
https://pluralistic.net/2024/05/15/they-trust-me-dumb-fucks/
This is the most charitable thing he has to say about AI.
> AI is a bubble and it will burst. Most of the companies will fail. Most of the data-centers will be shuttered or sold for parts. So what will be left behind?
> We'll have a bunch of coders who are really good at applied statistics. We'll have a lot of cheap GPUs, which'll be good news for, say, effects artists and climate scientists, who'll be able to buy that critical hardware at pennies on the dollar. And we'll have the open source models that run on commodity hardware, AI tools that can do a lot of useful stuff, like transcribing audio and video, describing images, summarizing documents, automating a lot of labor-intensive graphic editing, like removing backgrounds, or airbrushing passersby out of photos. These will run on our laptops and phones, and open source hackers will find ways to push them to do things their makers never dreamt of.
You can imagine that a guy who seriously thinks that the only thing AI will be doing in the future is summarising, describing images and transcribing is either completely clueless or deliberately misleading.
Not a person to be taken seriously

by LogicFailsMe

0 subcomment

For pennies on the dollar, we could just legalize and regulate psychedelics and anyone could go meet their god whenever they wish. The stoned ape theory might have been the AGI of spirituality that led to religion after all. Not saying it was, not saying it wasn't, but it's not like Elon Musk has to boil the ocean and build a Dyson Sphere to have a heart to heart with his personal invisible friend.
As for AI, it's incredibly useful in the right hands and it's incredibly hazardous in the wrong hands. But in the US, we can't even depose a lunatic flushing even more money than spent on AI on warmongering and you think we're gonna rein in the tech billionaires? Funny in that dying's easy it's comedy that's hard way. IMO this one plays out in the weakly efficient market of ELEs. My money's on DNA and planet Earth, it's been through so much worse and they always bounce back with new ideas on how to get in trouble again.
Not a doomer, AI and STEM could really deliver on the promise of a better future for everyone, but with tech billionaires driving the clown car, are you kidding me?

by minihat

7 subcomments

It's currently socially/politically unpalatable for authors to admit superintelligent AI is a possibility. I frequent some writer forums. As a group, they are 1) clearly feeling angry/threatened 2) in denial about LLM capabilities.
Folks working in software can more readily track progress of the frontier model performance.

by woeirua

3 subcomments

> I don't think AI is intelligent; nor do I think that the current (admittedly impressive) statistical techniques will lead to intelligence.
It’s increasingly difficult to rationalize away the capabilities of AI as not requiring “intelligence”. This point of view continues to require some belief in human exceptionalism.