- Great link. I've always been drawn to sqlite3 just from a simplicity and operational point of view. And with tools like "make it easy to replcate" Litestream and "make it easy to use" sqlite-utils, it just becomes easier.
And one of the first patterns I wanted to use was this. Just a read-only event log that's replicated, that is very easy to understand and operate. Kafka is a beast to manage and run. We picked it at my last company -- and it was a mistake, when a simple DB would have sufficed.
https://github.com/simonw/sqlite-utils
https://litestream.io/
- Peter (the author) is a really, really cool guy. We recorded a 3hr 30m podcast[0] with him a month ago. For anyone interested in the Kafka space, performance optimization in Rust and the general "why yet another Kafka", I'd shamelessly recommend the video:
[0] - https://www.youtube.com/watch?v=pJQ7hcsI1Dw
by 8organicbits
0 subcomment
- Quite cool. 7000 records per second is usable for a lot of projects.
One note on the backup/migrate, I think you need a shared lock on the database before you copy the database. If you dont, the database can corrupt. SQLite docs have other recommendations too:
https://sqlite.org/backup.html
- Any good and honest tansu experience reports out there? Would be nice to understand how “bleeding edge” this actually is, in practice. The idea of a kafka compatible, but trivial to run, system like this is very intriguing!
- I didn't know about Tansu and probably would not use it for anything too serious (yet!). Bus as a firm believer of event sourcing and change of paradigm that Kafka brings this is certainly interesting for small projects.
by krsoninikhil
1 subcomments
- I love sqlite backed system, one less component to worry about. But when using Tansu with sqlite storage, what are my options for horizonal scaling and keeping Tansu HA?
Also, are there any benchmark on how Tansu with S3 storage would perform in comparison to Kafka or something like WarpStream?
- To me it sounds like NATS Jetstream but with Rust. I wonder what the reliability looks like when it is prod ready
- everything is dead. what lives on is their protocol.
same for redis, kafka, ...
- How does it compare to Redis streams with persistent storage?
- This SQLite obsession is getting quite ridiculous. Now they put it in "the Cloud." What a shitshow. I wonder whether they know what SQLite is for... when Cloudflare did it, well, it made sense at least. This new generation of SQLite caro-culting is beyond anything I've ever seen.
Hacker News