To understand the Gradient Descent algorithm, just imagine you are blind and looking for the slope to reach the lowest point of the terrain you're on.

The backpropagation algorithm is even simpler.

Just do:

${\delta }_W=-\frac{\partial L}{\partial W_i}$ ${\delta }_W=-\left[\right(W_{i+1}^T\delta a_{i+1})\odot f_i^{\text{'}}(a_i\left)\right]h_{i-1}^T$ $\delta a_i=\frac{\partial L}{\partial a_i}$