Question 4 of 7: Least-squares parabola (proof) and a linear least-squares fit

Question 4: Least-squares parabola (proof) and a linear least-squares fit (A) & (B) 20 marks

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given (A). $n$ data points $(X_i,Y_i)$ presumed related by $Y=\alpha X+\beta X^2$ (no constant term).

Find (A). Prove the printed Cramer's-rule formulas for $\alpha,\beta$.

1. Minimize the sum of squared residuals. $S=\sum_{i=1}^n\big(Y_i-\alpha X_i-\beta X_i^2\big)^2$. Setting $\partial S/\partial\alpha=0$ and $\partial S/\partial\beta=0$: $$\dfrac{\partial S}{\partial\alpha}=-2\sum X_i(Y_i-\alpha X_i-\beta X_i^2)=0,\qquad \dfrac{\partial S}{\partial\beta}=-2\sum X_i^2(Y_i-\alpha X_i-\beta X_i^2)=0$$
2. Write the normal equations. $$\alpha\sum X_i^2+\beta\sum X_i^3=\sum X_iY_i,\qquad \alpha\sum X_i^3+\beta\sum X_i^4=\sum X_i^2Y_i$$ This is a $2\times2$ linear system $\begin{pmatrix}\sum X_i^2&\sum X_i^3\\\sum X_i^3&\sum X_i^4\end{pmatrix}\begin{pmatrix}\alpha\\\beta\end{pmatrix}=\begin{pmatrix}\sum X_iY_i\\\sum X_i^2Y_i\end{pmatrix}$.
3. Solve by Cramer's rule. With determinant $D=\big(\sum X_i^2\big)\big(\sum X_i^4\big)-\big(\sum X_i^3\big)^2$, $$\boxed{\alpha=\dfrac{\big(\sum X_iY_i\big)\big(\sum X_i^4\big)-\big(\sum X_i^3\big)\big(\sum X_i^2Y_i\big)}{D}},\qquad \boxed{\beta=\dfrac{\big(\sum X_i^2\big)\big(\sum X_i^2Y_i\big)-\big(\sum X_i^3\big)\big(\sum X_iY_i\big)}{D}}$$ which match the expressions printed on the exam page exactly.

Given (B). $X=(1,3,5,7,9,11)$, $Y=(66,52,49,35,23,18)$, model $Y=\alpha+\beta X$ (this time WITH an intercept, unlike part A).

Find (B). The least-squares $\alpha,\beta$.

1. Standard simple-linear-regression formulas. With $n=6$, $$\beta=\dfrac{n\sum X_iY_i-\sum X_i\sum Y_i}{n\sum X_i^2-(\sum X_i)^2},\qquad \alpha=\dfrac{\sum Y_i-\beta\sum X_i}{n}$$
2. Substitute the sums. $\sum X_i=36,\ \sum Y_i=243,\ \sum X_iY_i=1096,\ \sum X_i^2=286$: $$\beta=\dfrac{6(1096)-36(243)}{6(286)-36^2}=\dfrac{6576-8748}{1716-1296}=\dfrac{-2172}{420}=-4.87143$$ $$\alpha=\dfrac{243-(-4.87143)(36)}{6}=\dfrac{243+175.371}{6}=69.7286$$