> Overall, having spent a significant amount of time building this project, scaling it up to the size it’s at now, as well as analysing the data, the main conclusion is that it is not worth building your own solution, and investing this much time. When I first started building this project 3 years ago, I expected to learn way more surprising and interesting facts. There were some, and it’s super interesting to look through those graphs, however retrospectively, it did not justify the hundreds of hours I invested in this project.

The whole "quantified self" movement might be more about OCD and perfectionism than anything else.

/edit: quantified, not qualified

Paris accord says 1.5t per person per year, from all activities, Felix's flying alonre is ~10-15x current European yearly per person emissions and ~50-75x those compatible with +1.5C.

One pattern I have seen work well for the business version of this: a "company intelligence" database where everything known about a prospect company gets accumulated in one place over time. Homepage content, job postings, news mentions, funding history, tech stack signals, all deduplicated and queryable.

The challenge on the B2B side is the same as personal data: the data comes in from 8 different sources in 8 different formats, often with conflicts (two sources disagree on headcount, three sources have different founding dates). Your approach of controlling the schema from the start rather than trying to normalize later is the right call. Schema drift is what kills most long-term data projects.

What storage engine are you using? And how do you handle temporal data - do you snapshot state over time or just keep the latest version of each entity?

What's key is be able to visualize metrics easily on the data and frictionless data entry, I've got a decent setup with iPhone Action + Obsidian + QuickAdd scripts on Obsidian Sync (mobile + laptop). for visualization I use Obsidian Bases and Obsidian notes that run Dataview code blocks and Chart.js, couldn't be happier.

I could track things that are not interesting to reflect on like vitamin D supplementation for accountability but I've never bothered, especially if it's taken ~daily.

Every time I try to seriously track metrics of my life, the excitement of the insight gets worn away by the friction of recording and managing. I expect LLMs can help reduce the cost of this by an order of magnitude but then, as you mention, the question is, what do you do / change / learn because of the data?

I recently started tracking nutrition macros with an iOS app MacroFactor which I really like. This is the first time taking my weight doesn't feel like a IDK SHRUG moment and I can actually map my food intake to my weight.

Finances is probably the other highly actionable data source that is such high friction to manage (downloading CSVs, OFXs, monthly...) that it has always been a false start for me. I finally wrote a service to talk to Plaid directly and I successfully used it to categorize my business expenses at tax time. I finally have programmatic access to my bank account data!

You conclusion is definitely a cautionary take: > the main conclusion is that it is not worth building your own solution, and investing this much time.

But, perhaps a subset of that data you find useful.

Forgive me for I am being sceptical. The might be some insight here I have not considered, but I'd feel a lot more comfortable if it was all self-hosted / self-collected data.

I had the same epiphany as you days after acquiring a CO2 monitor. Most people notice poor indoor air quality from proxies such as humidity and temperature. AC (without ventilation) eliminates these and tricks our senses very effectively, giving us cool and fresh feeling indoor spaces full of CO2 and devoid of oxygen.

Most of the things that really hinder my productiveness and happiness are the same things that everyone tells you about; sleep, diet, sunlight, not being stuck indoors all day, socializing. And most of these can be improved by making changes to my habits.

Other than that, is that I think that it's important to get into the habit of self reflection. Having a feedback loop on my own life has helped me find out what works and what doesn't. It's too easy to just go through the days slogging through and not making any changes because you don't even realize what's going on.

I kept a rough log of my sleep and mood for about a year with no specific goal. Mostly forgot about it. Then I had a weirdly bad few months and went back to look — turns out there was a pretty clear pattern I would've never noticed in the moment.

Maybe the framing of "was it worth it" is the wrong question. It's less like an investment with a return and more like keeping receipts. Useless 99% of the time, then suddenly you really need one.

https://apps.apple.com/us/app/reflect-track-anything/id64638...

Another tangent, recently bought a paper shredder, started shredding through boxes of mail I have kept since they have personal info like cc statements but on the same idea of moving on/reducing stuff I'm hoarding whether it's data or physical.

I went through a similar process and came to the same conclusion, although with a slightly different twist. The project became the point and I almost stopped noticing the data towards the end.

Did you think of building some proactive AI tools to make use of all this centralized data?

https://edwardbetts.com/agenda/trip/past https://edwardbetts.com/agenda/trip/stats

This is why I moved to Tokyo. Even if I want to avoid exercising I still take many steps

380k datapoints sounds incredible but I imagine the real challenge is turning that into decisions that actually change behavior

I know this is the type of person i would not like.