MATH ML Learning Lab
A04
A04 · 20–30 min

SVD 与低秩近似SVD & Low-Rank Approximation

保留最重要的几个方向,能以多大误差重建矩阵?How accurately can a matrix be reconstructed from its most important directions?

前置知识:Prerequisites: A03:特征值与谱分解。A03: eigenvalues and spectral decomposition.

复习前置:Review prerequisites: A03 · 特征值与谱分解Eigenvalues & Spectral Decomposition

学完后你能:After this lesson you can: 按奇异值保留方向,并比较低秩重建误差。Retain directions by singular value and compare low-rank reconstruction error.

动手观察Explore

调整两个非负奇异值并选择保留项数,比较原矩阵、近似矩阵与重建误差。Adjust two nonnegative singular values and retained rank; compare the original matrix, approximation and reconstruction error.

从观察到解释From observation to explanation

分成方向贡献Split into directional contributions

SVD 将矩阵表示为多个秩一项之和。每项由一个奇异值及对应的左右方向决定;本实验使用对角矩阵,方向是两个坐标轴。SVD expresses a matrix as a sum of rank-one terms. Each term uses a singular value and corresponding left and right directions. This lab uses a diagonal matrix, whose directions are the coordinate axes.

保留最强方向Keep the strongest direction

对角矩阵 diag(3,1) 若只保留最大奇异值,近似为 diag(3,0),被丢弃项的 Frobenius 误差为 1;保留两项时精确重建。For diag(3,1), keeping only the largest singular value gives diag(3,0), with Frobenius error 1 from the discarded term. Keeping both reconstructs the matrix exactly.

适用于矩形矩阵Works for rectangular matrices

SVD 不限于方阵;保留较少方向可以压缩或近似。若矩阵本来秩亏,只需保留所有非零项即可重建;零矩阵没有非零方向。SVD also applies to rectangular matrices; keeping fewer directions gives compression or approximation. A rank-deficient matrix is reconstructed by all its nonzero terms; the zero matrix has no nonzero direction.

公式与符号Formula and symbols

A 是矩阵;σᵢ≥0 是从大到小排列的奇异值,uᵢ 和 vᵢ 是左、右奇异向量;Aₖ 只保留前 k 个非零方向。A is a matrix; σᵢ≥0 are singular values in descending order, and uᵢ and vᵢ are left and right singular vectors. Aₖ keeps the first k nonzero directions.

留意这个区别Watch for this

奇异值非负,和带符号的特征值不同。近似误差必须说明采用哪种误差度量。Singular values are nonnegative, unlike signed eigenvalues. State which error measure is used for approximation.

继续探索Explore further

继续学习算法Continue to an algorithm

主成分分析(PCA)Principal component analysis (PCA)

课后练习

检验你的理解