Hessian 与牛顿法Hessian & Newton’s Method
利用局部曲率，怎样计算一次牛顿步，何时会失败？How does local curvature define a Newton step, and when can it fail?
前置知识：Prerequisites: O01：二阶导数；B09：梯度；A03：特征值。O01: second derivatives; B09: gradients; A03: eigenvalues.
复习前置：Review prerequisites: O01 · 二阶导数与泰勒近似Second Derivatives & Taylor Approximation · B09 · 偏导数与梯度Partial Derivatives & Gradients · A03 · 特征值与谱分解Eigenvalues & Spectral Decomposition
学完后你能：After this lesson you can: 计算正定二次例子的牛顿步，并识别奇异与不定情形。Compute a Newton step for a positive quadratic and identify singular and indefinite cases.
动手观察Explore
选择正定、奇异或不定的二次曲面，比较梯度与一次牛顿步的结果。Choose a positive definite, singular or indefinite quadratic surface and compare the gradient with one Newton step.
从观察到解释From observation to explanation
二阶信息组成矩阵Second-order information as a matrix
对于 f(x,y)=1/2(ax²+by²)，梯度为 (ax,by)，Hessian 为对角矩阵 diag(a,b)。a、b 描述两个坐标方向的曲率。For f(x,y)=1/2(ax²+by²), the gradient is (ax,by) and the Hessian is diag(a,b). The entries a and b describe curvature along the two coordinate directions.
求一步Solve for a step
当 a、b 均为正时，解 HΔ=−∇f 得到 Δ=(−x,−y)，一步到达这个正定二次例子的最小点。这个结论依赖于例子的精确二次形状。When a and b are positive, solving HΔ=−∇f gives Δ=(−x,−y), reaching the minimum of this positive-definite quadratic in one step. This result depends on the exact quadratic example.
失败与区别Failure modes and a distinction
H 奇异时普通逆不存在；H 不定时一步可能到达鞍点而非最小点。Adam 的梯度平方指数平均是梯度的二阶原点矩估计，不是 Hessian。A singular H has no ordinary inverse; an indefinite H can lead to a saddle rather than a minimum. Adam’s exponential average of squared gradients estimates a second raw moment of gradients, not the Hessian.
公式与符号Formula and symbols
H 是由二阶偏导组成的 Hessian 矩阵，∇f 是梯度，Δ 是解线性方程得到的牛顿步，x 是参数向量；H 奇异时不能按普通可逆公式求步。H is the Hessian matrix of second partial derivatives, ∇f is the gradient, Δ solves a linear system for the Newton step, and x is the parameter vector. An ordinary inverse-based step is unavailable when H is singular.
留意这个区别Watch for this
牛顿步不自动保证下降；实际问题可能需要阻尼或其他处理。不要把 Adam 的梯度二阶矩当成损失的二阶导数。A Newton step does not automatically decrease the objective; real problems may need damping or other treatment. Do not equate Adam’s second gradient moment with second derivatives of the loss.
继续探索Explore further
课后练习
检验你的理解