Machine Learning Week 2 (Additional)

除了用梯度下降法，我们还有其他方法来求解

θ

的最小值：正规方程法（Normal Equation）

直接求导数。

θ \in R^{n + 1}

J (θ_{0}, θ_{1} \dots θ_{n}) = \frac{1}{2 m} \sum_{i = 1}^{m} (h_{θ} (x^{(i)}) - y^{(i)})^{2}

Let：

J (θ_{0}, θ_{1} \dots θ_{n}) = \frac{1}{2 m} \sum_{i = 1}^{m} (h_{θ} (x^{(i)}) - y^{(i)})^{2}

\frac{\partial}{\partial θ_{j}} J (θ) = \dots = 0

, For Every j

Solve

θ_{0}, θ_{1}, θ_{2}, \dots θ_{n}

θ_{0}, θ_{1}, θ_{2}, \dots θ_{n}

结果用矩阵表达是:

θ = (X^{T} X)^{- 1} X^{T} y

对于n较小的例子，求X的转置和逆都比较快，这样比梯度下降法快。但是如果n较大，比如10,000 左右那梯度下降法比较好。

附推导过程:

https://class.coursera.org/ml-008/forum/thread?thread_id=531

PS: Thank you, Mr. Anonymous ! If the solution derivation infringe on the author's copyright, I will delete that.

YG Overflow