there are a variety of rules of thumb (e.g. 5-9, 10 and 10-15 -- the guideline of 10 observations per independent variable is known as the "one in ten rule"). In the process of regression model selection, the mean squared error of the random regression function can be split into random noise, approximation bias, and variance in the estimate of the regression function. The bias-variance tradeoff is often used to overcome overfit models.

With a large