最小二乘系统辨识课 中篇：递归最小二乘

前言

本篇记录系统辨识课中的递归最小二乘原理。

最小二乘估计

如下的系统辨识模型：
$$\begin{gathered} \{\begin{array}{rl}y(1)& ={\phi }^T(1)\theta +v(1)\\ y(2)& ={\phi }^T(2)\theta +v(2)\\ & ...\\ y(t)& ={\phi }^T(t)\theta +v(t)\end{array} \\ \begin{array}{rl}设：Y_t& =[y(1)y(2)...y(t){]}^T\\ H_t& =[\phi (1)\phi (2)...\phi (t){]}^T\\ V_t& =[v(1)v(2)...v(n){]}^T\end{array} \\ 有：Y_t=H_t\theta +V_t \end{gathered}$$

参数的最小二乘估计可表示为：
$${\hat{\theta }}_{LS}=(H_t^T{H}_t{)}^{-1}{H}_t^T{Y}_t$$

上面的方法在每个新时刻的输入输出数据更新时，都要重估计，计算负载很大。

递归最小二乘估计

原理

递归最小二乘估计采用了增量思路，推导如下：

$$\begin{gathered} P^{-1}(t)=H_t^T{H}_t=\sum _{i=1}^t\phi (i){\phi }^T(i)=\sum _{i=1}^{t-1}\phi (i){\phi }^T(i)+\phi (t){\phi }^T(t)=P^{-1}(t-1)+\phi (t){\phi }^T(t) \\ \begin{array}{rl}\hat{\theta }(t)& =(H_t^T{H}_t{)}^{-1}{H}_t^T{Y}_t\\ & =P(t)H_t^T{Y}_t\\ & =P(t)\left[\begin{array}{cc}{H}_{t-1}^T& \phi (t)\end{array}\right]{\left[\begin{array}{cc}{Y}_{t-1}^T& y(t)\end{array}\right]}^T\\ & =P(t)H_{t-1}^T{Y}_{t-1}^T+P(t)\phi (t)y(t)\\ & =P(t)P^{-1}(t-1)P(t-1)H_{t-1}^T{Y}_{t-1}^T+P(t)\phi (t)y(t)\\ & =P(t)P^{-1}(t-1)\hat{\theta }(t-1)+P(t)\phi (t)y(t)\\ & =P(t)[P^{-1}(t)-\phi (t){\phi }^T(t)]\hat{\theta }(t-1)+P(t)\phi (t)y(t)\\ & =\hat{\theta }(t-1)-P(t)\phi (t){\phi }^T(t)\hat{\theta }(t-1)+P(t)\phi (t)y(t)\\ & =\hat{\theta }(t-1)+P(t)\phi (t)[y(t)-{\phi }^T(t)\hat{\theta }(t-1)]\end{array} \\ 引理：矩阵A,B,C，若(I+C{A}^{-1}B)与A都可逆，则： \\ (A+BC{)}^{-1}=A^{-1}-A^{-1}B(I+C{A}^{-1}B{)}^{-1}C{A}^{-1} \\ {P}^{-1}(t)=P^{-1}(t-1)+\phi (t){\phi }^T(t) \\ 把后面的式子取逆，根据引理有： \\ P(t)=P(t-1)-\frac{P(t-1)\phi (t){\phi }^T(t)P(t-1)}{1+{\phi }^T(t)P(t-1)\phi (t)} \\ P(t)\phi (t)=\frac{P(t-1)\phi (t)}{1+{\phi }^T(t)P(t-1)\phi (t)} \\ L(t)=P(t)\phi (t) \end{gathered}$$

于是，系统参数的递归最小二乘可表示为：
$$\begin{gathered} \hat{\theta }(t)=\hat{\theta }(t-1)+L(t)[y(t)-{\phi }^T(t)\hat{\theta }(t-1)]] \\ L(t)=\frac{P(t-1)\phi (t)}{1+{\phi }^T(t)P(t-1)\phi (t)} \\ P(t)=[I-L(t){\phi }^T(t)]P(t-1),P(0)=p_{0}I,p_0=\inf \\ \phi (t)=[...] \end{gathered}$$

新息，残差

新息$e(t)=y(t)-{\phi }^T(t)\hat{\theta }(t-1)$
残差$\epsilon (t)=y(t)-{\phi }^T(t)\hat{\theta }(t)$

数据饱和

现象

当输入输出时间序列足够长时，新的输入输出对递归最小二乘估计结果没有贡献的现象，称为数据饱和。

原因

$$\begin{gathered} P(t)=[P^{-1}(t-1)+\phi (t){\phi }^T(t){]}^{-1}=>P(t)\le P(t-1) \\ (\alpha I+1/p_0)t\le P^{-1}(t)=P^{-1}(0)+\sum _{i=1}^t\phi (i){\phi }^T(i)\le (\beta I+1/p_0)t \\ \underset{t->\inf }{\lim }{P}^{-1}(t)=0 \end{gathered}$$

解决方法：带遗忘因子的递归最小二乘

$$\begin{gathered} \hat{\theta }(t)=\hat{\theta }(t-1)+L(t)[y(t)-{\phi }^T(t)\hat{\theta }(t-1)]] \\ L(t)=\frac{P(t-1)\phi (t)}{\lambda +{\phi }^T(t)P(t-1)\phi (t)} \\ P(t)=\frac{1}{\lambda }[I-L(t){\phi }^T(t)]P(t-1),P(0)=p_{0}I,p_0=\inf \\ \phi (t)=[...] \end{gathered}$$

后记

本篇记录了递归最小二乘的推导，数据饱和现象以及解决方法。下篇将记录最小二乘中，${\phi }^T(t)$的获取方法。