Bootstrap

Hadamard积 学习笔记(张贤达《矩阵分析与应用》)

本文是张贤达老师《矩阵分析与应用(第2版)》的部分阅读笔记。

定义

m × n m \times n m×n矩阵 A = [ a i j ] \boldsymbol{A}=\left[a_{i j}\right] A=[aij] m × n m \times n m×n矩阵 B = [ b i j ] \boldsymbol{B}=\left[b_{i j}\right] B=[bij]的Hadamard积记作 A ⊙ B \boldsymbol{A} \odot \boldsymbol{B} AB,它仍然是一个 m × n m \times n m×n矩阵,定义为: A ⊙ B = [ a i j b i j ] \boldsymbol{A} \odot \boldsymbol{B}=\left[a_{i j} b_{i j}\right] AB=[aijbij]Hadamard积也称Schur积或者对应元素乘积。

与正定性有关的性质

正定性的传递性

如果 m × m m \times m m×m维矩阵 A , B \boldsymbol{A},\boldsymbol{B} A,B是正定的(半正定)的,则他们的Hadamard积也是正定(半正定的)

正定性的反推(Fejer定理)

我们可反推 m × m m \times m m×m维矩阵 A \boldsymbol{A} A是半正定矩阵,当且仅当 ∑ i = 1 m ∑ j = 1 m a i j b i j ⩾ 0 \sum_{i=1}^{m} \sum_{j=1}^{m} a_{i j} b_{i j} \geqslant 0 i=1mj=1maijbij0对所有 m × m m \times m m×m维半正定矩阵 B \boldsymbol{B} B成立。

与矩阵迹有关的性质

定理1

A , B , C \boldsymbol{A},\boldsymbol{B},\boldsymbol{C} A,B,C m × n m \times n m×n矩阵,并且 1 = [ 1 , 1 , ⋯   , 1 ] T \boldsymbol{1}=[1,1,\cdots,1]^\mathrm{T} 1=[1,1,,1]T n × 1 n \times 1 n×1求和向量, D = d i a g ( d 1 , d 2 , ⋯   , d m ) \boldsymbol{D}=\mathrm{diag}(d_1,d_2,\cdots,d_m) D=diag(d1,d2,,dm),其中, d i = ∑ j = 1 n a i j d_{i}=\sum_{j=1}^{n} a_{i j} di=j=1naij,则 tr ⁡ ( A T ( B ⊙ C ) ) = tr ⁡ ( ( A T ⊙ B T ) C ) \operatorname{tr}\left(\boldsymbol{A}^{\mathrm{T}}(\boldsymbol{B} \odot \boldsymbol{C})\right)=\operatorname{tr}\left(\left(\boldsymbol{A}^{\mathrm{T}} \odot \boldsymbol{B}^{\mathbf{T}}\right) \boldsymbol{C}\right) tr(AT(BC))=tr((ATBT)C) 1 T A T ( B ⊙ C ) 1 = tr ⁡ ( B T D C ) \mathbf{1}^{\mathbf{T}} \boldsymbol{A}^{\mathrm{T}}(\boldsymbol{B} \odot \boldsymbol{C}) \mathbf{1}=\operatorname{tr}\left(\boldsymbol{B}^{\mathrm{T}} \boldsymbol{D} \boldsymbol{C}\right) 1TAT(BC)1=tr(BTDC)

定理2

A , B \boldsymbol{A},\boldsymbol{B} A,B n × n n \times n n×n正方矩阵,并且 1 = [ 1 , 1 , ⋯   , 1 ] T \boldsymbol{1}=[1,1,\cdots,1]^\mathrm{T} 1=[1,1,,1]T n × 1 n \times 1 n×1求和向量。假定 M \boldsymbol{M} M是一个 n × n n \times n n×n对角矩阵 M = d i a g ( μ 1 , μ 2 , ⋯   , μ n ) \boldsymbol{M}=\mathrm{diag}(\mu_1,\mu_2,\cdots,\mu_n) M=diag(μ1,μ2,,μn),而 m = M 1 \boldsymbol{m}=\boldsymbol{M1} m=M1 n × 1 n \times 1 n×1向量,则有: tr ⁡ ( A M B T M ) = m T ( A ⊙ B ) m tr ⁡ ( A B T ) = 1 T ( A ⊙ B ) 1 M A ⊙ B T M = M ( A ⊙ B T ) M \begin{aligned} \operatorname{tr}\left(\boldsymbol{A} \boldsymbol{M} \boldsymbol{B}^{\mathrm{T}} \boldsymbol{M}\right) &=\boldsymbol{m}^{\mathrm{T}}(\boldsymbol{A} \odot \boldsymbol{B}) \boldsymbol{m} \\ \operatorname{tr}\left(\boldsymbol{A} \boldsymbol{B}^{\mathrm{T}}\right) &=\mathbf{1}^{\mathrm{T}}(\boldsymbol{A} \odot \boldsymbol{B}) \mathbf{1} \\ \boldsymbol{M} \boldsymbol{A} \odot \boldsymbol{B}^{\mathrm{T}} \boldsymbol{M} &=\boldsymbol{M}\left(\boldsymbol{A} \odot \boldsymbol{B}^{\mathrm{T}}\right) \boldsymbol{M} \end{aligned} tr(AMBTM)tr(ABT)MABTM=mT(AB)m=1T(AB)1=M(ABT)M

一般性质

  1. A , B \boldsymbol{A},\boldsymbol{B} A,B均为 m × n m \times n m×n矩阵,则 A ⊙ B = B ⊙ A ( A ⊙ B ) T = A T ⊙ B T ( A ⊙ B ) H = A H ⊙ B H ( A ⊙ B ) ∗ = A ∗ ⊙ B ∗ \begin{aligned} \boldsymbol{A} \odot \boldsymbol{B} &=\boldsymbol{B} \odot \boldsymbol{A} \\ (\boldsymbol{A} \odot \boldsymbol{B})^{\mathrm{T}} &=\boldsymbol{A}^{\mathrm{T}} \odot \boldsymbol{B}^{\mathrm{T}} \\ (\boldsymbol{A} \odot \boldsymbol{B})^{\mathrm{H}} &=\boldsymbol{A}^{\mathrm{H}} \odot \boldsymbol{B}^{\mathrm{H}} \\ (\boldsymbol{A} \odot \boldsymbol{B})^{*} &=\boldsymbol{A}^{*} \odot \boldsymbol{B}^{*} \end{aligned} AB(AB)T(AB)H(AB)=BA=ATBT=AHBH=AB
  2. 任意矩阵与零矩阵的Hadamard积: A ⊙ O m × n = O m × n ⊙ A = O m × n \boldsymbol{A} \odot \boldsymbol{O}_{m \times n}=\boldsymbol{O}_{m \times n} \odot \boldsymbol{A}=\boldsymbol{O}_{m \times n} AOm×n=Om×nA=Om×n
  3. 若c为常数,则 c ( A ⊙ B ) = ( c A ) ⊙ B = A ⊙ ( c B ) c(\boldsymbol{A} \odot \boldsymbol{B})=(c \boldsymbol{A}) \odot \boldsymbol{B}=\boldsymbol{A} \odot(\boldsymbol{c} \boldsymbol{B}) c(AB)=(cA)B=A(cB)
  4. 正定(或半正定)矩阵 A , B \boldsymbol{A},\boldsymbol{B} A,B的Hadamard积 A ⊙ B \boldsymbol{A} \odot \boldsymbol{B} AB也是正定(或半正定的)。
  5. 矩阵 A m × m = [ a i j ] \boldsymbol{A}_{m \times m}=\left[a_{i j}\right] Am×m=[aij]与单位矩阵 I m \boldsymbol{I}_m Im的Hadamard积为 m × m m \times m m×m对角矩阵,即: A ⊙ I m = I m ⊙ A = diag ⁡ ( A ) = diag ⁡ ( a 11 , a 22 , ⋯   , a m m ) \boldsymbol{A} \odot \boldsymbol{I}_{m}=\boldsymbol{I}_{m} \odot \boldsymbol{A}=\operatorname{diag}(\boldsymbol{A})=\operatorname{diag}\left(a_{11}, a_{22}, \cdots, a_{m m}\right) AIm=ImA=diag(A)=diag(a11,a22,,amm)
  6. A , B , C , D \boldsymbol{A},\boldsymbol{B},\boldsymbol{C},\boldsymbol{D} A,B,C,D均为 m × n m \times n m×n矩阵,则 A ⊙ ( B ⊙ C ) = ( A ⊙ B ) ⊙ C = A ⊙ B ⊙ C ( A ± B ) ⊙ C = A ⊙ C ± B ⊙ C ( A + B ) ⊙ ( C + D ) = A ⊙ C + A ⊙ D + B ⊙ C + B ⊙ D \begin{aligned} \boldsymbol{A} \odot(\boldsymbol{B} \odot \boldsymbol{C}) &=(\boldsymbol{A} \odot \boldsymbol{B}) \odot \boldsymbol{C}=\boldsymbol{A} \odot \boldsymbol{B} \odot \boldsymbol{C} \\ (\boldsymbol{A} \pm \boldsymbol{B}) \odot \boldsymbol{C} &=\boldsymbol{A} \odot \boldsymbol{C} \pm \boldsymbol{B} \odot \boldsymbol{C} \\ (\boldsymbol{A}+\boldsymbol{B}) \odot(\boldsymbol{C}+\boldsymbol{D}) &=\boldsymbol{A} \odot \boldsymbol{C}+\boldsymbol{A} \odot \boldsymbol{D}+\boldsymbol{B} \odot \boldsymbol{C}+\boldsymbol{B} \odot \boldsymbol{D} \end{aligned} A(BC)(A±B)C(A+B)(C+D)=(AB)C=ABC=AC±BC=AC+AD+BC+BD
  7. A , B , D \boldsymbol{A},\boldsymbol{B},\boldsymbol{D} A,B,D m × m m \times m m×m矩阵,且 D \boldsymbol{D} D为对角矩阵,则 ( D A ) ⊙ ( B D ) = D ( A ⊙ B ) D (\boldsymbol{D}\boldsymbol{A})\odot(\boldsymbol{B}\boldsymbol{D})=\boldsymbol{D}(\boldsymbol{A}\odot\boldsymbol{B})\boldsymbol{D} (DA)(BD)=D(AB)D
  8. A , C \boldsymbol{A},\boldsymbol{C} A,C m × m m \times m m×m矩阵,且 B , D \boldsymbol{B},\boldsymbol{D} B,D n × n n \times n n×n矩阵。则 ( A ⊕ B ) ⊙ ( C ⊕ D ) = ( A ⊙ C ) ⊕ ( B ⊙ D ) (\boldsymbol{A} \oplus \boldsymbol{B}) \odot(\boldsymbol{C} \oplus \boldsymbol{D})=(\boldsymbol{A} \odot \boldsymbol{C}) \oplus(\boldsymbol{B} \odot \boldsymbol{D}) (AB)(CD)=(AC)(BD)
  9. A , B , C \boldsymbol{A},\boldsymbol{B},\boldsymbol{C} A,B,C m × n m \times n m×n矩阵,则 tr ⁡ ( A T ( B ⊙ C ) ) = tr ⁡ ( ( A T ⊙ B T ) C ) \operatorname{tr}\left(\boldsymbol{A}^{\mathrm{T}}(\boldsymbol{B} \odot \boldsymbol{C})\right)=\operatorname{tr}\left(\left(\boldsymbol{A}^{\mathrm{T}} \odot \boldsymbol{B}^{\mathrm{T}}\right) \boldsymbol{C}\right) tr(AT(BC))=tr((ATBT)C)

Hadamard积服从的不等式

  1. Oppenheim不等式:令 A \boldsymbol{A} A B \boldsymbol{B} B n × n n \times n n×n 半正定矩阵,则 ∣ A ⊙ B ∣ ⩾ a 11 ⋯ a n n ∣ B ∣ |\boldsymbol{A} \odot \boldsymbol{B}| \geqslant a_{11} \cdots a_{n n}|\boldsymbol{B}| ABa11annB特例:若 B = I n \boldsymbol{B}=\boldsymbol{I}_n B=In,且 A \boldsymbol{A} A n × n n \times n n×n半正定矩阵,则有Hadamard不等式: ∣ A ∣ ≤ a 11 ⋯ a n n |\boldsymbol{A}|\leq a_{11}\cdots a_{nn} Aa11ann这是因为 ∣ A ∣ = b 11 ⋯ b n n ∣ A ∣ ⩽ ∣ I n ⊙ A ∣ |\boldsymbol{A}|=b_{11} \cdots b_{n n}|\boldsymbol{A}| \leqslant\left|\boldsymbol{I}_{n} \odot \boldsymbol{A}\right| A=b11bnnAInA,而 I n ⊙ A = diag ⁡ ( a 11 , ⋯   , a n n ) \boldsymbol{I}_{n} \odot \boldsymbol{A}=\operatorname{diag}\left(a_{11}, \cdots, a_{n n}\right) InA=diag(a11,,ann),故而有 ∣ A ∣ ≤ a 11 ⋯ a n n |\boldsymbol{A}|\leq a_{11}\cdots a_{nn} Aa11ann
  2. A \boldsymbol{A} A B \boldsymbol{B} B n × n n \times n n×n 半正定矩阵,则 ∣ A ⊙ B ∣ ⩾ ∣ A B ∣ |\boldsymbol{A} \odot \boldsymbol{B}| \geqslant|\boldsymbol{A} \boldsymbol{B}| ABAB
  3. 特征值不等式:令 A \boldsymbol{A} A B \boldsymbol{B} B n × n n \times n n×n 半正定矩阵, λ 1 , ⋯   , λ n \lambda_{1}, \cdots, \lambda_{n} λ1,,λn是Hadamard积 A ⊙ B \boldsymbol{A} \odot \boldsymbol{B} AB的特征值,而 λ ^ 1 , ⋯   , λ ^ n \hat{\lambda}_{1}, \cdots, \hat{\lambda}_{n} λ^1,,λ^n是矩阵乘积 A B \boldsymbol{A}\boldsymbol{B} AB的特征值,则 ∏ i = k n λ i ⩾ ∏ i = k n λ ^ i , k = 1 , ⋯   , n \prod_{i=k}^{n} \lambda_{i} \geqslant \prod_{i=k}^{n} \hat{\lambda}_{i}, \quad k=1, \cdots, n i=knλii=knλ^i,k=1,,n可以理解为不等式2的特征值版本。
  4. Hadamard积的秩不等式:令 A \boldsymbol{A} A B \boldsymbol{B} B n × n n \times n n×n 矩阵,则 rank ⁡ ( A ⊙ B ) ⩽ rank ⁡ ( A ) rank ⁡ ( B ) \operatorname{rank}(\boldsymbol{A} \odot \boldsymbol{B}) \leqslant \operatorname{rank}(\boldsymbol{A}) \operatorname{rank}(\boldsymbol{B}) rank(AB)rank(A)rank(B)
;