| 2008 | |
| 2009 | |
| 2010 | class FullyConnected(LayerBase): |
| 2011 | def __init__(self, n_out, act_fn=None, init="glorot_uniform", optimizer=None): |
| 2012 | r""" |
| 2013 | A fully-connected (dense) layer. |
| 2014 | |
| 2015 | Notes |
| 2016 | ----- |
| 2017 | A fully connected layer computes the function |
| 2018 | |
| 2019 | .. math:: |
| 2020 | |
| 2021 | \mathbf{Y} = f( \mathbf{WX} + \mathbf{b} ) |
| 2022 | |
| 2023 | where `f` is the activation nonlinearity, **W** and **b** are |
| 2024 | parameters of the layer, and **X** is the minibatch of input examples. |
| 2025 | |
| 2026 | Parameters |
| 2027 | ---------- |
| 2028 | n_out : int |
| 2029 | The dimensionality of the layer output |
| 2030 | act_fn : str, :doc:`Activation <numpy_ml.neural_nets.activations>` object, or None |
| 2031 | The element-wise output nonlinearity used in computing `Y`. If None, |
| 2032 | use the identity function :math:`f(X) = X`. Default is None. |
| 2033 | init : {'glorot_normal', 'glorot_uniform', 'he_normal', 'he_uniform'} |
| 2034 | The weight initialization strategy. Default is `'glorot_uniform'`. |
| 2035 | optimizer : str, :doc:`Optimizer <numpy_ml.neural_nets.optimizers>` object, or None |
| 2036 | The optimization strategy to use when performing gradient updates |
| 2037 | within the :meth:`update` method. If None, use the :class:`SGD |
| 2038 | <numpy_ml.neural_nets.optimizers.SGD>` optimizer with |
| 2039 | default parameters. Default is None. |
| 2040 | |
| 2041 | Attributes |
| 2042 | ---------- |
| 2043 | X : list |
| 2044 | Running list of inputs to the :meth:`forward <numpy_ml.neural_nets.LayerBase.forward>` method since the last call to :meth:`update <numpy_ml.neural_nets.LayerBase.update>`. Only updated if the `retain_derived` argument was set to True. |
| 2045 | gradients : dict |
| 2046 | Dictionary of loss gradients with regard to the layer parameters |
| 2047 | parameters : dict |
| 2048 | Dictionary of layer parameters |
| 2049 | hyperparameters : dict |
| 2050 | Dictionary of layer hyperparameters |
| 2051 | derived_variables : dict |
| 2052 | Dictionary of any intermediate values computed during |
| 2053 | forward/backward propagation. |
| 2054 | """ # noqa: E501 |
| 2055 | super().__init__(optimizer) |
| 2056 | |
| 2057 | self.init = init |
| 2058 | self.n_in = None |
| 2059 | self.n_out = n_out |
| 2060 | self.act_fn = ActivationInitializer(act_fn)() |
| 2061 | self.parameters = {"W": None, "b": None} |
| 2062 | self.is_initialized = False |
| 2063 | |
| 2064 | def _init_params(self): |
| 2065 | init_weights = WeightInitializer(str(self.act_fn), mode=self.init) |
| 2066 | |
| 2067 | b = np.zeros((1, self.n_out)) |
no outgoing calls