FRESH

Hacker News

The early History of the Singular Value Decomposition (1993) [pdf]

53 points by wolfi1

by muragekibicho

2 subcomments

For the curious, eigenvalues only exist for square matrices. Singular values are like generalized eigenvalues.
Singular values are like the fundamental frequencies of your matrix. You know how you can define any color with RGB? In a (pretty handwavy) way, singular values are like RGB color codes for us math guys.
Optimizers like Muon and Adam play around with weights' first, or second order singular values to train models.

by waynecochran

3 subcomments

The SVD seems to come up everywhere in my work in computer vision. I find myself continuously using the various C++/Eigen SVD implementations. Actually I should speak in the past tense. Claude and Codex are now generating all my code for me now, and I see them spitting out SVD code frequently -- often for very special cases. SVD truly is an amazing tool.