Note that it won't help you if your workload makes use of all your RAM at once.

If you have a bunch of stuff running in the background it will help a lot.

I get 2 to 3 compression factor at all times with zstd. I calculated the utility to be as if I had 20GB extra RAM for what I do.

1. https://x.com/_asadmemon/status/1989417143398797424

Now, almost everything on the server side is a VM or a container. We have lots of neighbors who want to share the CPU and the RAM, and the RAM is the bigger constraint because the CPUs have 192 cores and each of those cores does a dozen times as much work as a decade ago. Heck, we used to have the memory controller on the motherboard and the last level of cache was a chip or module of SRAM outside the CPU.

We also have a situation now in which the multiple in speed of the CPU over RAM has skyrocketed, but the caches have gotten far larger and much smarter. Smaller things arranged differently in RAM make things run faster because they make better use of the cache.

Now that RAM is expensive, shared, and program and data size and arrangements are bound to cache behavior, optimization can lean heavily into optimizing for RAM again.

Some of these arguments hold true for desktop systems as well.

I have wondered for years when the time will come that instead of such huge and smart caches, someone will just put basically register-speed RAM on the chip and swap to motherboard RAM the way we swap to disk. HBM is somewhere close, being a substrate stacked in the package but not in the CPU die itself.

It reminds me of the heady days of Thai floods when hard drives were inaccessible.

One thing that might support this is the fact AI companies are purchasing uncut wafers of DRAM. One use might be to hoard and stockpile them somewhere in a cave, so that no one else gets to them.

Another thing that might support this is that precisely the same strategy had been in use by software companies during the COVID hiring fever. Companies used to hire people for ridiculous pay with little actual work to perform so that among other things, competitors wouldn’t whisk those people away and be at an advantage.

This, of course, ended with massive layoffs once the reckoning came about, and I’m wondering about what is going to happen when (there’s no “if”) the reckoning comes for big AI, too.

Raise your hand if you have been there too! :-))

Unless there is a true breakthrough, beyond AGI into super intelligence on existing, or near term, hardware— I just don’t see how “trust me bro,” can keep its spending party going. Competition is incredibly stiff, and it’s pretty likely we’re at the point of diminishing returns without an absolute breakthrough.

The end result is going to be RAM prices tanking in 18-24 months. The only upside will be for consumers who will likely gain the ability to run much larger open source models locally.

2010s: so much memory, programmers used electron and chrome wrapping everything in js.

2026: so little memory, programmers have to optimize AI code to run properly.

We can't get any new chips. At all. We can't launch our new product because nobody could afford the memory even if we could get some.

Incredible.

Behold, the RAM cost is being optimized with AI.

People is missing the point. Mega-corporations distort the market. This is not capitalism this is old aristocratic ruling by power. If all these monopolies were divided in smaller chunks and regulated to not allow them to abuse that power we will not be here.

This situation is not normal, big tech is currently above the law and above the market economy and if they fail their plan is to make us pay *AGAIN* for their bad decisions. All businesses and individuals are already paying higher prices for big tech folly, we will be left with the bill when the AI boom fails, too.

It denied this saying that the figures quoted were estimates only, that such massive RAM contracts would be easily obtainable public knowledge and that primarily the recent price increases were mostly cyclical in nature.

Any truth to this?

Edit to add: I am actually curious; I was under the impression that this 40% story going around was true and confirmed, rather than just hyperbole or speculation.