Chapter 3 Matrix decompositions

This chapter focuses on two ways to decompose a matrix into smaller parts. We can then think about which are the most important parts of the matrix, and that will be useful when we think about dimension reduction. The highlight of the chapter is the singular value decomposition (SVD), which is one of the most useful mathematical concepts from the past century, and is relied upon throughout statistics and machine learning. The SVD extends the idea of the eigen (or spectral) decomposition of symmetric square matrices to any matrix.

• Matrix-matrix products
• Eigenvalues and the spectral decomposition
• Introduction to the singular value decomposition
• SVD optimization results
• Low-rank approximation