Regression Models: Building Simple Models from Sample Data

See Jain, Chapters 14 and 15

What should I know?

• Response variable vs. predictor variables

• Motivation for regression models:
  • Investigate dependence of one variable on others.
  • Extrapolate.

• Types of regression models:
  • Simplest case: linear with one predictor
  • Multiple linear
  • Nonlinear

• How to compute a regression model --- use software tools! For example, Excel, SAS, JMP, ExpertFit.

• Meaning of ``least squares fit.'' If the model is , where the function is defined by k parameters , then the least squares fit corresponds to choosing the parameters to minimize

  

• Allocation of variation. The coefficient of determination is defined as

  

  where

  

• Computing confidence intervals for regression parameters (see 14.5):
  • From the standard deviation of the sample errors, we can compute ...
  • standard deviations for each regression parameter, from which we can compute ...
  • confidence intervals for the ``true'' parameters, (typically using a t-distribution).

• Computing confidence intervals for predictions (future values), see 14.6.

• Importance of visual tests for goodness of fit. See 14.7.

• What about non-numerical predictor variables? See 15.2.

• A note on polynomial models.
  • If the model is y = a x^b, then do simple linear regression on log(y) = log(a) + b log(x).
  • If the model is y = a + bx^m (for assumed m), then do simple linear regression on y = a + b (x^m).