An insufficiently complex model won’t explain the data well. An overly complex model will overfit the training data.
A process to find the right model:
- Start with a low-order model, like a linear model
- Look at not only the r-squared value, but how well it accounts for new data
- Increase the order of the model, and repeat
- Keep doing this until a model does a good job on the training data and predicting new data