Store key to the item in a database. Then you can query it however you want.
I could imagine it though that S3 could offer something similar. We can already list the bucket items, why not add some of querying ability?
by gberger
2 subcomments
S3: Simple Storage Service. It's a building block, and it's only natural other abstractions are built on top of it.
by CodesInChaos
1 subcomments
The "dataset" abstraction this article proposes feels rather specific for their use-case, not universal. None of my S3 use-cases would benefit from it.
Just store such metadata in your database, where you can organize, index and aggregate it whatever way you like.
by hilariously
1 subcomments
The prefixes are not meaningless, they are performance boundaries as well if you read the docs.
by skybrian
1 subcomments
Why do they put everything into one huge bucket? Wouldn't the best way to clean it up be to create more buckets?
by dchess
1 subcomments
I feel like this is what delta lake and ducklake are largely solving for. And then some.
Hacker News