It is assumed that the cause and effect relationship between the variables remains unchanged. This assumption may not always hold good and hence estimation of the values of a variable made on the basis of the regression equation may lead to erroneous and misleading results.

We often see patterns or relationships in scatterplots. When the y variable tends to increase as the x variable increases, we say there is a positive correlation between the variables. When the y variable tends to decrease as the x variable increases, we say there is a negative correlation between the variables.

What are the three limitations of correlation and regression? Because although 2 variables may be associated with each other, they may not necessarily be causing each other to change. In other words, a lurking variable may be present. Why does association not imply causation?

(a) Limitations of Bivariate Regression: (i) Linear regression is often inappropriately used to model non-linear relationships (due to lack in understanding when linear regression is applicable). (ii) Linear regression is limited to predicting numeric outputs only. It gives no qualitative information.

Strengths: Linear regression is straightforward to understand and explain, and can be regularized to avoid overfitting. In addition, linear models can be updated easily with new data using stochastic gradient descent. Weaknesses: Linear regression performs poorly when there are non-linear relationships.

It is sensitive to outliers and poor quality data—in the real world, data is often contaminated with outliers and poor quality data. If the number of outliers relative to non-outlier data points is more than a few, then the linear regression model will be skewed away from the true underlying relationship.