What exactly does delta mean in the gradient descent algorithm?

Question

As on the picture:

Could someone help me understand what exactly what delta means in the gradient descent algorithm?

Answer 1

The term is a derivative with respect to the theta 0.

• Mark theta as coordinate on X-axis (let it be A)
• Find corresponding coordinate on Y-axis (let it be B) so the point belongs to the function J
• Draw tangent line to that function at the point (A, B)
• The derivative is the slope of this tangent line.

The derivative is used to control two aspects of the cost function (J function) minimization:

• direction - sign of the slope tells you in which direction you should move along the X-axis in order to converge J
• rate - magnitude of the slope tells you how fast you should move

Answer 2

This is a partial derivative with respect to theta_0.