Fall 2003 

You have seen the method to fit data (x_{1},y_{1}),..., (x_{m},y_{m}) by
functions of the form f=a_{1} f_{1}(x)+ ... + a_{n} f_{n}(x).
The idea was to write a system A a = b of linear equations which tell
that all the data can be fitted by the functions f(x_{i})=y_{i}, then
find the least square solution
a_{*} = (A^{T} A)^{1} A^{T} b to this system. (The least square solution formula was obtained from A^{T} (bA a) = 0 which paraphrases that the "error" (bA a) is perpendicular to the image of A. Geometrically this means that A a is the projection of b onto the image of A. ) 
Examples. Linear fitting with a linear function
f(x) = a_{1} x + a_{2}, where n=2:
(The same formula can also be obtained using Lagrange extrema by minimizing Var[a_{1} X + a_{2}  Y] under the constraint E[a_{1} X + a_{2} Y]=0.) 
We can also fit data (x_{1},y_{1},z_{1}), ..., (x_{m},y_{m},z_{m})in three dimensions by functions z=f(x,y) = a_{1} f_{1}(x,y) + ... + a_{n} f_{n}(x,y) or implicitly by surfaces g(x,y,z) = 0 like for example g(x,y,z) = a_{1}x^{2} + a_{2} y^{2} + a_{3} z^{2} + a_{4}, in which case one would want to find the best ellipsoid centered at the origin which fits the three dimensional data. 