Question 4 of 6: Image Restoration

Question 4: Image Restoration (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. Four canonical degradation/recovery pairs: noise/denoising, missing-region/inpainting, blur/deblurring, downsampling/super-resolution.

Find. For each: a realistic context, the degradation's mathematical model, and a recovery strategy.

(a) Noise / Denoising

Context. A photograph taken in low light (high sensor ISO gain amplifies read noise), or an ultrasound/laser image corrupted by speckle (Question 3(e)).

Mathematical formulation. Additive noise: $g(x,y)=f(x,y)+n(x,y)$, with $n$ typically modelled zero-mean; multiplicative noise (speckle) instead follows $g=f\cdot n$.

Recovery. If multiple independent noisy observations of the same static scene are available (e.g. repeated frames), averaging $N$ frames reduces the noise variance by a factor of $N$ (averaged over blocks of $N=10$, has its variance drop by a factor of about $12.5\times$, consistent with the $1/N$ law); for a single image, spatial low-pass/median filtering (Question 2(d)) or wavelet shrinkage (Question 2(e)) trade a controlled amount of detail loss for noise reduction.

(b) Removing Image Pieces / Inpainting

Context. Removing a scratch or a date-stamp burned into an old photograph, or removing an unwanted object/watermark from an image.

Mathematical formulation. A binary mask $M(x,y)$ marks the removed region ($M=1$ inside the hole): $g(x,y)=f(x,y)\,\big(1-M(x,y)\big)$, and the recovery problem is to estimate $\hat f(x,y)$ for every pixel where $M=1$, given only $g$ on $M=0$.

Recovery. For small/thin gaps, propagate structure inward from the boundary along isophotes (lines of constant intensity) via a PDE-based diffusion (e.g. solving a Laplace-type equation with the known boundary as a Dirichlet condition); for larger regions with repeating texture elsewhere in the image, exemplar/patch-based methods search the rest of the image for well-matching patches and copy them in, which handles texture far better than pure diffusion.

(c) Blurring / Deblurring

Context. Camera or subject motion during exposure (motion blur), or an out-of-focus lens (defocus blur).

Mathematical formulation. Blur is convolution with a point-spread function $h$ (plus, realistically, additive noise): $g(x,y)=f(x,y)*h(x,y)+n(x,y)$, equivalently $G(u,v)=F(u,v)H(u,v)+N(u,v)$ in the frequency domain (the same convolution theorem invoked for fast filtering in Question 2(c)).

Recovery. If $h$ is known (or estimated), a regularized inverse/Wiener filter $\hat F=\dfrac{H^{*}G}{|H|^2+K}$ recovers $f$ while limiting the noise amplification that a naive $G/H$ division would suffer wherever $H$ is small; if $h$ is unknown, blind deconvolution jointly estimates $h$ and $f$ under a statistical/parametric prior on one or both.

(d) Downsampling / Super-Resolution

Context. Generating a thumbnail or a lower-resolution preview from a high-resolution photo, or reconstructing a high-resolution still from a lower-resolution video/scanner feed.

Mathematical formulation. Ideal downsampling by factor $D$ is point sampling of a properly band-limited signal, $g[n]=f[Dn]$; the degradation being modelled here is naive downsampling that skips band-limiting first, which by the sampling theorem aliases any content above the new Nyquist frequency $f_s/(2D)$ into the low-frequency band.

Recovery. The correct forward operation is not actually invertible after the fact (information above the new Nyquist rate is genuinely destroyed by proper anti-alias filtering, or corrupted if it wasn't filtered) — so "recovery" here means designing the downsampling correctly in the first place (low-pass filter to the new Nyquist limit before decimating), or, for upsampling/super-resolution, using multiple lower-resolution observations with sub-pixel shifts (or a learned image prior) to statistically estimate detail beyond what any single low-resolution frame contains. This was verified quantitatively: for a signal containing both a low-frequency component (bin 3 of 64) and a higher-frequency component (bin 11) chosen so that naive decimation-by-8 folds bin 11 onto bin 3, naive point-decimation reproduces the low-frequency target with an error of exactly $2.0$ (fully corrupted by the aliased component's own amplitude of 2.0), while decimating after an ideal anti-alias low-pass filter (zeroing every frequency bin outside the new Nyquist band before downsampling) reproduces the target to numerical precision ($<10^{-13}$).