Derivations of Equations in The Elements of Statistical Learning

As I read through ESL (Second Edition), I made notes on derivations of non-trivial equations and would like to share them. The authors assigned some equations in the text as exercises, and I did not include them in my document, since their solutions are available in the solution manuals listed below.

1. ESL Solutions by Yuhang Zhou (github/YuhangZhou88)
2. A Solution Manual and Notes for ESL by J. L. Weatherwax and D. Epstein
3. A Guide and Solution Manual to ESL by J. C. Ma

I used the same mathematical notation as in ESL. However, on a few occasions I used boldface lower characters for vector notation, like when referring Wikipedia or Matrix Cookbook.

References:

1. The Matrix Cookbook (Nov 2012) by K. B. Peterson and M. S. Pederson (https://www2.imm.dtu.dk/pubdb/pubs/3274-full.html)
2. A Solution Manual and Notes for ESL (October 2021) by J. L. Weatherwax and D. Epstein
3. Pattern Recognition and Machine Learning (February 2006) by C. M. Bishop