MCPcopy
hub / github.com/ddbourgin/numpy-ml / backward

Method backward

numpy_ml/neural_nets/layers/layers.py:1572–1604  ·  view source on GitHub ↗

Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray ` of shape `(n_ex, in_rows, in_cols, in_ch)` The gradient of the loss wrt. the layer output `Y`. retain_grads : bool Whether to incl

(self, dLdy, retain_grads=True)

Source from the content-addressed store, hash-verified

1570 return y
1571
1572 def backward(self, dLdy, retain_grads=True):
1573 """
1574 Backprop from layer outputs to inputs.
1575
1576 Parameters
1577 ----------
1578 dLdY : :py:class:`ndarray <numpy.ndarray>` of shape `(n_ex, in_rows, in_cols, in_ch)`
1579 The gradient of the loss wrt. the layer output `Y`.
1580 retain_grads : bool
1581 Whether to include the intermediate parameter gradients computed
1582 during the backward pass in the final parameter update. Default is
1583 True.
1584
1585 Returns
1586 -------
1587 dX : :py:class:`ndarray <numpy.ndarray>` of shape `(n_ex, in_rows, in_cols, in_ch)`
1588 The gradient of the loss wrt. the layer input `X`.
1589 """ # noqa: E501
1590 assert self.trainable, "Layer is frozen"
1591 if not isinstance(dLdy, list):
1592 dLdy = [dLdy]
1593
1594 dX = []
1595 X = self.X
1596 for dy, x in zip(dLdy, X):
1597 dx, dScaler, dIntercept = self._bwd(dy, x)
1598 dX.append(dx)
1599
1600 if retain_grads:
1601 self.gradients["scaler"] += dScaler
1602 self.gradients["intercept"] += dIntercept
1603
1604 return dX[0] if len(X) == 1 else dX
1605
1606 def _bwd(self, dy, X):
1607 """Computation of gradient of the loss wrt X, scaler, intercept"""

Callers 1

test_LayerNorm2DFunction · 0.95

Calls 1

_bwdMethod · 0.95

Tested by 1

test_LayerNorm2DFunction · 0.76