Rectangular system: QR decomposition and least squares

When there are more equations than variables, one can't hope for an exact solution, the system is overdetermined.

We try to minimize the error which is usually measured as the norm

$$\vert\vert Ax-b\vert\vert,\; for\; the\; equation\; Ax=b$$

Important remark: For any orthogonal Q, this norm is unchanged by trnasformation by $Q$, so

$$\vert\vert Ax-b\vert\vert=\vert\vert Q'Ax-Q'b\vert\vert$$

we try and find a $Q$ that makes the minimization particularly simple, that is so that $Q'A$ has a particularly simple form.