Class NetworkGraves

cdslib/core/nn/modules/graves.py:162–565

The Alex Graves' model in https://arxiv.org/abs/1308.0850 The network includes (see Fig 12 in the paper): - attn_rnn - gaussian sliding window attention - decode_rnn - output layer It does not include (since they are application-independent): - loss - samplin

class NetworkGraves(nn.Module):
    """
    The Alex Graves' model in https://arxiv.org/abs/1308.0850

    The network includes (see Fig 12 in the paper):

    - attn_rnn
    - gaussian sliding window attention
    - decode_rnn
    - output layer

    It does not include (since they are application-independent):

    - loss
    - sampling method

    Methods:
        1. forward

            - inputs:

                - [x0, x1, ..., x_{T-1}]
                - [c1, c2, ..., cN]
                - initial hidden states
                - (optional) zt_fun that generates zt from current states

            - outputs:

                - [p1, p2, ..., p_T]
                - [\hat{c1}, ... \hat{cT}]
                - final hidden states

        2. get_all_zero_hidden_states

    """

    def __init__(self, param_dict: ParamGraves = None, **kwargs):
        """Create an Alex Graves' model.

        Args:
            param_dict:
                A :py:class:`ParamGraves` object to define the hyper-parameters of the network.
            kwargs:
                If param_dict is None, you can directly provide keyword arguments of :py:class:`ParamGraves` here.
        """
        super().__init__()

        # read and set configs
        if param_dict is not None:
            self.config_dict = ParamGraves(**param_dict)
        else:
            self.config_dict = ParamGraves(**kwargs)

        for key in self.config_dict:
            setattr(self, key, self.config_dict[key])

        # construct sub-networks
        self._construct_networks()