An insufficiently complex model won’t explain the data well. An overly complex model will overfit the training data.

A process to find the right model:

  • Start with a low-order model, like a linear model
  • Look at not only the r-squared value, but how well it accounts for new data
  • Increase the order of the model, and repeat
  • Keep doing this until a model does a good job on the training data and predicting new data