Least-squares identification of a first-order discrete model: 22-Elec-B2 Advanced Control Systems, May 2015, Q3

Question 3: Least-squares identification of a first-order discrete model (25 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. Seven noisy samples of a single-input single-output system driven by a unit pulse, to be fitted with a first-order discrete model.

Quantity	Value
Model structure	$P(z) = \dfrac{\beta}{z - \alpha}$, i.e. $y(k+1) = \alpha\,y(k) + \beta\,u(k)$
Output samples $y(0)\ldots y(6)$	$0,\ 10,\ 4,\ 3,\ 1.6,\ 0.4,\ 0.3$
Input samples $u(0)\ldots u(6)$	$1,\ 0,\ 0,\ 0,\ 0,\ 0,\ 0$
Noise	zero mean, white, additive on the measurements
Part (b) input	$u(k) = 2$ held constant

Find. The least-squares estimates of $\alpha$ and $\beta$, and the steady-state output the identified model predicts for a constant input of 2.

Question 3: the seven measured samples (red) against the least-squares model y(k+1) = 0.4507 y(k) + 10 u(k) driven by the same unit pulse (blue).

Approach. Write the difference equation as a linear regression in the two unknown parameters, assemble as many equations as the data actually support, and solve the normal equations; the structure of this particular input makes the two estimates independent.

Part (a) — put the model in regression form. Cross-multiplying $Y(z)(z-\alpha) = \beta U(z)$ and inverting gives the difference equation $$y(k+1) = \alpha\,y(k) + \beta\,u(k) = \underbrace{\begin{pmatrix}y(k) & u(k)\end{pmatrix}}_{\varphi^{T}(k)} \underbrace{\begin{pmatrix}\alpha\\ \beta\end{pmatrix}}_{\vartheta},$$ which is linear in the unknown parameters even though the system itself is dynamic. This is what makes ordinary least squares applicable.
Count the equations the data really provide. Each equation needs $y(k)$, $u(k)$ and $y(k+1)$, so the last sample can only ever appear on a left-hand side. Seven samples therefore yield six equations, not seven: $$\begin{aligned} k=0:&\quad 10 = \alpha(0) + \beta(1) &\qquad k=3:&\quad 1.6 = \alpha(3) + \beta(0)\\ k=1:&\quad 4 = \alpha(10) + \beta(0) &\qquad k=4:&\quad 0.4 = \alpha(1.6) + \beta(0)\\ k=2:&\quad 3 = \alpha(4) + \beta(0) &\qquad k=5:&\quad 0.3 = \alpha(0.4) + \beta(0) \end{aligned}$$ Six equations in two unknowns is heavily over-determined, which is the whole reason a least-squares fit is asked for rather than a pair of exact solutions.
Assemble the regression matrices. Stacking the six rows, $$\Phi = \begin{pmatrix}0 & 1\\ 10 & 0\\ 4 & 0\\ 3 & 0\\ 1.6 & 0\\ 0.4 & 0\end{pmatrix}, \qquad Y = \begin{pmatrix}10\\ 4\\ 3\\ 1.6\\ 0.4\\ 0.3\end{pmatrix}, \qquad \hat\vartheta = \bigl(\Phi^{T}\Phi\bigr)^{-1}\Phi^{T}Y .$$
Form the normal equations. The cross term vanishes because the input is non-zero only at $k = 0$, where the output happens to be zero as well: $$\Phi^{T}\Phi = \begin{pmatrix}\sum y(k)^2 & \sum y(k)u(k)\\ \sum y(k)u(k) & \sum u(k)^2\end{pmatrix} = \begin{pmatrix}127.72 & 0\\ 0 & 1\end{pmatrix}, \qquad \Phi^{T}Y = \begin{pmatrix}57.56\\ 10\end{pmatrix},$$ with $\sum y(k)^2 = 10^2 + 4^2 + 3^2 + 1.6^2 + 0.4^2 = 127.72$ and $\sum y(k)y(k+1) = 40 + 12 + 4.8 + 0.64 + 0.12 = 57.56$.
Solve for the two parameters. The diagonal normal matrix decouples the estimates completely, so each is read off its own row: $$\boxed{\;\hat\beta = \frac{10}{1} = 10, \qquad \hat\alpha = \frac{57.56}{127.72} = 0.4507\;}$$ The pulse gain is recovered exactly from the single equation that contains it, while the pole estimate is the least-squares average of the five decay ratios.
Check the residuals against the noise assumption. Substituting the estimates back, $$e = Y - \Phi\hat\vartheta = \begin{pmatrix}0 & -0.507 & +1.197 & +0.248 & -0.321 & +0.120\end{pmatrix}^{T}.$$ They alternate in sign, sum to approximately zero and show no trend, which is consistent with the stated zero-mean white measurement noise. Had they shown a systematic pattern, the first-order model structure itself would have been in doubt.
Part (b) — check stability before quoting a DC gain. A steady state exists only if the identified pole lies inside the unit circle, and $|\hat\alpha| = 0.4507 \lt 1$, so it does. Setting $y(k+1) = y(k) = y_{ss}$ and $u(k) = \bar u$ in the difference equation, $$y_{ss} = \hat\alpha\,y_{ss} + \hat\beta\,\bar u \;\Longrightarrow\; y_{ss} = \frac{\hat\beta}{1 - \hat\alpha}\,\bar u = \hat P(1)\,\bar u,$$ which is simply the model evaluated at $z = 1$.
Evaluate the steady-state output. The DC gain is $$\hat P(1) = \frac{10}{1 - 0.4507} = 18.204,$$ so a constant input of 2 gives $$\boxed{\;y_{ss} = 2 \times 18.204 = 36.41\;}$$

Question 3 — results
Quantity	Symbol	Value
Equations available from 7 samples	—	6
Regression sums	$\sum y^2,\ \sum y(k)y(k{+}1)$	127.72, 57.56
Pole estimate	$\hat\alpha$	0.4507
Gain estimate	$\hat\beta$	10
Identified model	$\hat P(z)$	$\dfrac{10}{z - 0.4507}$
Largest residual	$\max\|e_k\|$	1.197 (at $k = 2$)
DC gain	$\hat P(1)$	18.204
(b) Steady-state output at $u = 2$	$y_{ss}$	36.41

Topic. Linear least-squares identification of a difference-equation model from input-output records.

Key relations. $y(k+1) = \varphi^{T}(k)\vartheta$; $\hat\vartheta = (\Phi^{T}\Phi)^{-1}\Phi^{T}Y$; residual $e = Y - \Phi\hat\vartheta$; DC gain $\hat P(1) = \hat\beta/(1 - \hat\alpha)$; the discrete pole $\hat\alpha = e^{-h/\tau}$ maps back to a continuous time constant $\tau = -h/\ln\hat\alpha$.

Why this works. Although the plant is dynamic, the unknown parameters enter the difference equation linearly, so the fit is an ordinary linear regression. Because $\Phi^{T}\Phi$ turns out diagonal here, the two estimates do not interact and each reduces to a one-line quotient; in general they would have to be solved together.

Common pitfall. Writing seven equations instead of six — the final sample supplies a left-hand side but no regressor row, and including a fabricated seventh row biases the estimate. A second pitfall is solving two of the equations exactly rather than in the least-squares sense: taking $k = 1$ and $k = 2$ alone gives $\alpha = 0.4$ and $\alpha = 0.75$, which disagree precisely because the data are noisy. A third is quoting a DC gain without first confirming $|\hat\alpha| \lt 1$; for an unstable estimate no steady state exists.

Source. L. Ljung, System Identification: Theory for the User, 2nd ed., ch. 7 (the least-squares estimate and its properties); Franklin, Powell & Workman, Digital Control of Dynamic Systems, 3rd ed., ch. 12 (identification for digital control).

Question 3 of 6: Least-squares identification of a first-order discrete model

Question 3: Least-squares identification of a first-order discrete model (25 marks)