There are wealthy gentlemen in England who drive four-horse passenger-coaches twenty or thirty miles on a daily line in the summer because the privilege costs them considerable money; but if they were offered wages for the service, that would turn it into work, and then they would resign.

-- Mark Twain

Only a few percent of your judgements are ever tested (by surviving being optioned, produced and released) and, of the ones that are, at best you only get a small sampling of false positives over a sea of potential false negatives. I imagine he's incredibly interested in the fate of any titles he didn't recommend which end up being produced (perhaps by another studio). Having filled a similar role in a different industry with similar high-stakes 'unknowables', I thought a lot about this. It was pretty obvious what practically mattered was how much my output "felt right" to downstream decision-makers vs actually being right.

While my stakeholders were quite happy with my work, actually targeting such ephemeral and uncorrelated feedback felt unproductive and dumb. Eventually, I settled on making the evaluation process fully transparent and consistent. I ensured all objective criteria were documented and each subjective judgement had clear confidence intervals. This was more challenging than it sounds. In the end, it was still hard to know if I was really improving year to year. For that, I still had to rely on my own, mostly subjective, self-assessment but at least I had some objective tracking data to calibrate on. That at least helped me feel like I was executing with diligence and integrity. It also increased my confidence no one else in the industry was doing it any better.

[1] https://www.youtube.com/watch?v=Nb2tebYAaOA&t=5039s

What surprises me is that he only reads about 50 more books a year than I do, and he does it full time.

https://youtube.com/@mortismalgaming

Not for him though, he loves it.