Normal Equation in Linear Regression

Original Source: https://www.coursera.org/learn/machine-learning

In linear regression we’ve talked about, we found optimum theta by iterating with gradient descent.

The “Normal Equation” is a method of finding the optimum theta without iteration.

\[\theta = (X^T X)^{-1}X^Ty\]

Comparison of gradient descent and normal equation

Gradient Descent	Normal Equation
Needs to choose alpha	No need to choose alpha
Needs many iterations	No need to iterate
Works well when n is large	Slow if n is very large