$\because \nabla f(x)=2Ax+2b$
$\nabla f(x+\alpha d)=2A(x+\alpha d)+2b$
又 $\because \frac{ \partial h(\alpha)}{\partial (\alpha)} = \nabla f(x+\alpha d) \frac {\partial (x+\alpha d)}{\partial \alpha} =\nabla f(x+\alpha d)d=0$
$\therefore \alpha=-\frac {d^\intercal (2b+2Ax)}{2d^\intercal Ad}$
$=-\frac {d^\intercal \nabla f(x)}{2d^\intercal Ad}$

1 二阶范数

$$||A||^2 = tr(AA^T) \quad\quad\quad\quad\quad\quad\quad\quad$$
2 矩阵求导基本公式（等式两边同时去除tr，公式不变）：
$$tr(A) = tr(A^T) \quad\quad\quad\quad\quad\quad\quad\quad$$
$$\frac{\partial tr(AX)}{\partial X} = A^T \quad\quad\quad\quad\quad\quad\quad\quad$$

$$\frac{\partial tr(AX)}{\partial X^T} = A \quad\quad\quad\quad\quad\quad\quad\quad$$

$$\frac{\partial tr(AXB)}{\partial X } = (BA)^T = A^TB^T \quad\quad\quad\quad\quad\quad\quad\quad$$
3 则简单公式均可以推导出来：
$$\frac{\partial tr(X^TAX)}{\partial X} = AX + (X^TA)^T = (A^T+A)X \quad\quad\quad\quad\quad\quad\quad\quad$$
$$\frac{\partial tr(X^TAX)}{\partial X^T} = (AX)^T + X^TA = X^T(A^T+A) \quad\quad\quad\quad\quad\quad\quad\quad$$
$$\frac{\partial tr(ABA^TC)}{\partial A} = (BA^TC)^T+CAB=C^TAB^T+CAB \quad\quad\quad\quad\quad\quad\quad\quad$$

1.伴随矩阵

1.定义

设$A=[a_{ij}]$是$n$阶矩阵，行列式$\left|A\right|$的每个元素$a_{ij}$的代数余子式$A_{ij}$所构成的如下的矩阵
$$A^{\ast }=\left[\begin{array}{cccc}{A}_{11}& A_{21}& \cdots & A_{n1}\\ A_{12}& A_{22}& \cdots & A_{n2}\\ ⋮& ⋮& & ⋮\\ A_{1n}& A_{2n}& \cdots & A_{nn}\end{array}\right]$$
称为矩阵$A$的伴随矩阵.

2.二阶矩阵的逆矩阵

对于2阶矩阵，用主对角线元素对换，副对角线元素变号即可求出伴随矩阵。
$$A^{\ast }=\left[\begin{array}{cc}{A}_{11}& A_{21}\\ A_{12}& A_{22}\end{array}\right]=\left[\begin{array}{cc}d& -b\\ -c& a\end{array}\right]$$

3.公式

$$\begin{gathered} A{A}^{{}_{{}_{\ast }}}=A^{{}_{{}_{\ast }}}A=\left|A\right|E; \\ {A}^{{}_{{}_{\ast }}}=\left|A\right|A^{-1};\left|A^{{}_{{}_{\ast }}}\right|={\left|A\right|}^{n-1}; \\ {\left(A^{\ast }\right)}^{-1}={\left(A^{-1}\right)}^{\ast }=\frac{1}{|A|}A; \\ {\left(A^{\ast }\right)}^T={\left(A^T\right)}^{\ast };{\left(kA\right)}^{\ast }=k^{n-1}{A}^{{}_{{}_{\ast }}};{\left(A^{\ast }\right)}^{\ast }={\left|A\right|}^{n-2}A; \\ r(A^{\ast })=\{\begin{array}{l}n,如果r(A)=n,\\ 1,如果r(A)=n-1,\\ 0,如果r(A)<n-1.\end{array} \end{gathered}$$

2.逆矩阵

1.定义

设$A$是$n$阶矩阵，如果存在是$n$阶矩阵$B$使得$AB=BA=E$（单位矩阵）成立，则称$A$是可逆矩阵或非奇异矩阵，$B$是$A$的逆矩阵。

2.定理

1. 若$A$是可逆矩阵，则矩阵$A$的逆矩阵唯一，记为$A^{-1}$.

2. $$\begin{gathered} n阶矩阵A可逆 \\ \iff \left|A\right|\ne 0 \\ \iff r(A)=n \\ \iff A的列(行)向量组线性无关 \\ \iff A=P_1{P}_2\cdots P_s{P}_i(i=1,2,\cdots ,s)是初等矩阵 \\ \iff A与单位矩阵等价 \\ \iff 0不是矩阵A的特征值 \end{gathered}$$

3. 若$A$是$n$阶矩阵，则满足$AB=E$，则必有$BA=E$

3.公式

$$\begin{gathered} {\left(A^{-1}\right)}^{-1}=A;{\left(kA\right)}^{-1}=\frac{1}{k}{A}^{-1}\left(k\ne 0\right); \\ {\left(AB\right)}^{-1}=B^{-1}{A}^{-1};{\left(A^n\right)}^{-1}={\left(A^{-1}\right)}^n; \\ {\left(A^{-1}\right)}^T={\left(A^T\right)}^{-1};\left|A^{-1}\right|=\frac{1}{|A|}; \\ {A}^{-1}=\frac{1}{|A|}{A}^{\ast } \end{gathered}$$

3.作业

5、矩阵求逆函数

In [1]: import numpy as np

In [2]: a = np.array([[-2, 3, 3], [1, -1, 0], [-1, 2, 1]])

In [3]: np.linalg.inv(a)
Out[3]:
array([[-0.5,  1.5,  1.5],
       [-0.5,  0.5,  1.5],
       [ 0.5,  0.5, -0.5]])

In [4]: A = np.matrix(a)

In [5]: A.I
Out[5]:
matrix([[-0.5,  1.5,  1.5],
        [-0.5,  0.5,  1.5],
        [ 0.5,  0.5, -0.5]])

In [6]: b = np.array([[0, 3, 3], [1, 1, 0], [-1, 2, 3]])

In [7]: a.dot(b)
Out[7]:
array([[ 0,  3,  3],
       [-1,  2,  3],
       [ 1,  1,  0]])