The query itself represents information. If you can anticipate 100% of the ways in which you intend to query the information (no surprises), I'd argue there might be an ideal way to store it.

* For lossless compression of generic data, gzip or zstd.

* For text, documentation, and information without fancy formatting, markdown, which is effectively a plain-text superset.

* For small datasets, blobs, objects, and what not, JSON.

* For larger datasets and durable storage, SQLite3.

Whenever there's text involved, use UTF-8. Whenever there's dates, use ISO8601 format (UTC timezone) or Unix timestamps.

Following these rules will keep you happy 80% of the time.

I doubt we humans will be able to do better (faster, more capacity, more analytical, more intuitive, more logical) storage (at an individual level, not at mass scale, since that's kinda achieved already by the behemoths like Google, etc.) in a few thousand years of civilization.

Quantum computing may be the game changer though.

I read somewhere that the entirety of humanity's information, including all knowledge and data of past (of every human ever) and current, if stored via quantum computing - that quanta of quantum information will just be the size of a football.

Conceptually similar to CAP, but with storage trade-offs. The idea is you can only pick 2 out of 3.

I've been thinking about trade-offs as "pick two of three" in the abstract, but the bookshelf example made it concrete. The insight that matters is: if you know your query patterns, you can optimize differently.

As a PM, I keep trying to build systems that work for "every case." But this article reminded me that's the wrong goal. The hash table works because it accepts the space-time trade-off. The heap works because it embraces disorder for non-priority items.

Sometimes the best system isn't the most elegant one—it's the one that matches how you'll actually use it.

Good reminder to stop over-optimizing for flexibility I'll never need.

Thanks for sharing.