Inference on a Least-Squares Linear Regression: 23-Chem-B3 Simulation, Modelling, and Optimization, December 2014, Q5

Question 5: Inference on a Least-Squares Linear Regression (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. $n = 10$ pairs; $b_0 = -0.8701$, $b_1 = 8.5168$; $S_{{yy}} = 80.61$ (total), $SS_R = 76.87$ (regression); error variance $s^2 = 0.468$; $C_{{00}} = 1.72911$, $C_{{11}} = 0.94354$. Critical two-sided $t$ values at $\nu = n-2 = 8$ d.o.f.:

$\alpha$	0.50	0.40	0.30	0.20	0.10
$t(\alpha/2)$	0.706	0.889	1.108	1.397	1.860

Find. (a) test $H_0:\beta_1 = 0$; (b) test $H_0:\beta_0 = 0$ (whether the through-origin model suffices); (c) the correlation coefficient $r$.

Approach. Use $s = \sqrt{{s^2}}$ and standard errors $\operatorname{{se}}(b_j) = s\sqrt{{C_{{jj}}}}$ to form $t = b_j/\operatorname{{se}}(b_j)$ on $\nu=8$ d.o.f.; obtain $r$ from the variance partition $r^2 = SS_R/S_{{yy}}$.

Residual scale and d.o.f. $s = \sqrt{{0.468}} = 0.6841$ on $\nu = n-2 = 8$ d.o.f. (Check: $SS_E = S_{{yy}} - SS_R = 80.61 - 76.87 = 3.74$, and $SS_E/8 = 0.468$ — consistent with the stated error variance.)
(a) Test $H_0:\beta_1 = 0$. The slope $t$-statistic is $$t = \frac{b_1}{s\sqrt{{C_{{11}}}}} = \frac{8.5168}{0.6841\sqrt{{0.94354}}} = \frac{8.5168}{0.6645} = \boxed{12.82}.$$ Since $12.82$ vastly exceeds every tabulated critical value (largest $= 1.860$ at $\alpha=0.10$), reject $H_0$: $Y$ depends significantly on $x$.
(b) Test $H_0:\beta_0 = 0$. The intercept $t$-statistic is $$t = \frac{b_0}{s\sqrt{{C_{{00}}}}} = \frac{-0.8701}{0.6841\sqrt{{1.72911}}} = \frac{-0.8701}{0.8996} = \boxed{-0.967}.$$ Because $|{-0.967}| \lt 1.860$ (and indeed sits between the $\alpha=0.40$ and $\alpha=0.30$ criticals), we fail to reject $H_0:\beta_0=0$. The intercept is not significant, so the through-origin model $Y = \beta_1 x$ can replace the original.
(c) Correlation coefficient. From the variance partition, $$r^2 = \frac{SS_R}{S_{{yy}}} = \frac{76.87}{80.61} = 0.9536 \;\Rightarrow\; r = +\sqrt{{0.9536}} = \boxed{0.9765},$$ positive because $b_1 \gt 0$.

Test / quantity	Statistic	Decision
(a) $H_0:\beta_1 = 0$	$t = 12.82$ ($\nu=8$)	Reject — $Y$ depends on $x$
(b) $H_0:\beta_0 = 0$	$t = -0.967$	Fail to reject — $Y=\beta_1 x$ acceptable
(c) Correlation coefficient	$r = +0.9765$	$r^2 = 0.954$

Concept & Theory

Topic. Hypothesis testing and interval inference for ordinary least-squares (OLS) regression coefficients.

Key relations. $\hat{{\boldsymbol\beta}} = (X^{{T}}X)^{{-1}}X^{{T}}Y$; $\operatorname{{Var}}(\hat\beta_j) = s^2 C_{{jj}}$; $t_j = b_j/(s\sqrt{{C_{{jj}}}})\sim t_{{n-2}}$; variance partition $S_{{yy}} = SS_R + SS_E$ and $r^2 = SS_R/S_{{yy}}$.

Why this works. The diagonal of $(X^{{T}}X)^{{-1}}$ scaled by the residual variance gives each coefficient's variance, so a coefficient divided by its own standard error is a $t$ variate. A large slope-$t$ says the predictor matters; a small intercept-$t$ says the line may be forced through the origin without loss. The coefficient of determination is exactly the fraction of total variability the regression explains.

Common pitfall. Using $n$ or $n-1$ instead of $\nu = n-2$ d.o.f. for simple regression, or dropping the sign of $r$ (it inherits the sign of $b_1$).

Source. Montgomery & Runger, Applied Statistics and Probability for Engineers, ch. 11 (simple linear regression, tests on coefficients, $r^2$).

Question 5 of 7: Inference on a Least-Squares Linear Regression

Question 5: Inference on a Least-Squares Linear Regression (20 marks)