CS3414 Afterclass Notes --- 4 June, 2002
Fitting Data
- Least Squares Approximation of Data
- Basic idea: approximate given set of m data points,
(xi,yi), by a
function F(x) that is the `best fit' to that data, in some sense.
- Important questions:
- What should F(x) look like? How will you represent it?
Using what basis functions, for example?
- What do you mean by `best fit'?
- The `least squares' answer to the 2nd question is that we
want the F(x) that minimizes
r12 +
r22 + ... +
rm2, where
ri = yi - F(xi).
(Terminology: ri is the ith `residual').
- There are other reasonable answers, e.g., we could minimize
the 1-norm or the infinity-norm of the residual vector. In
practice, the least squares (2-norm) is most common because
solving the minimization problem is easier and because it has nice
statistical properties.