backpropagation
an archive of posts with this tag
-
Understanding Backpropagation Through Time (BPTT)
BPTT backpropagates gradients through temporal dependencies in an RNN. This post explains how BPTT works and how to compute its gradients.
-
Backpropagation through convolution: deriving kernel and input gradients
Derive the kernel and input gradients of a convolutional layer using the chain rule, and connect cross-correlation, full convolution, kernel flipping, and stamp accumulation.
-
Softmax and Cross-Entropy
Deriving the softmax Jacobian and the cross-entropy gradient, and why their composition collapses into prediction minus target.