Initialize network weights `W` using the He normal initialization strategy. Notes ----- The He normal initialization strategy initializes the weights in `W` using draws from TruncatedNormal(0, b) where the variance `b` is .. math:: b = \\frac{2}{\\text{fan_in}}
(weight_shape)
| 893 | |
| 894 | |
| 895 | def he_normal(weight_shape): |
| 896 | """ |
| 897 | Initialize network weights `W` using the He normal initialization strategy. |
| 898 | |
| 899 | Notes |
| 900 | ----- |
| 901 | The He normal initialization strategy initializes the weights in `W` using |
| 902 | draws from TruncatedNormal(0, b) where the variance `b` is |
| 903 | |
| 904 | .. math:: |
| 905 | |
| 906 | b = \\frac{2}{\\text{fan_in}} |
| 907 | |
| 908 | He normal initialization was originally developed for deep networks with |
| 909 | :class:`~numpy_ml.neural_nets.activations.ReLU` nonlinearities. |
| 910 | |
| 911 | Parameters |
| 912 | ---------- |
| 913 | weight_shape : tuple |
| 914 | The dimensions of the weight matrix/volume. |
| 915 | |
| 916 | Returns |
| 917 | ------- |
| 918 | W : :py:class:`ndarray <numpy.ndarray>` of shape `weight_shape` |
| 919 | The initialized weights. |
| 920 | """ |
| 921 | fan_in, fan_out = calc_fan(weight_shape) |
| 922 | std = np.sqrt(2 / fan_in) |
| 923 | return truncated_normal(0, std, weight_shape) |
| 924 | |
| 925 | |
| 926 | def glorot_uniform(weight_shape, gain=1.0): |
nothing calls this directly
no test coverage detected