EagerParamBase is derived from Tensor( Which is the concept in Eager-Dygraph Mode). A EagerParamBase is a persistable Tensor, and will be updated by optimizers after each iteration. The training of a neural network is essentially the updating of its EagerParamBase. Relative
| 7794 | |
| 7795 | |
| 7796 | class EagerParamBase(core.eager.Tensor): |
| 7797 | """ |
| 7798 | EagerParamBase is derived from Tensor( Which is the concept in Eager-Dygraph Mode). |
| 7799 | A EagerParamBase is a persistable Tensor, and will be updated by optimizers |
| 7800 | after each iteration. |
| 7801 | The training of a neural network is essentially the updating of |
| 7802 | its EagerParamBase. |
| 7803 | |
| 7804 | Relative to a general Tensor, a EagerParamBase has several its own |
| 7805 | member variables: |
| 7806 | |
| 7807 | Args: |
| 7808 | trainable(bool): True if the EagerParamBase need to be updated after |
| 7809 | iterations. |
| 7810 | optimize_attr(map): EagerParamBase attributes related with optimizing. |
| 7811 | Currently, it only contains 'learning_rate'. |
| 7812 | Default: {'learning_rate': 1.0} |
| 7813 | regularizer(WeightDecayRegularizer): The Regularizer which will |
| 7814 | be applied on the EagerParamBase. Default: None |
| 7815 | do_model_average(bool): True if the model average strategy will |
| 7816 | be applied on this EagerParamBase. |
| 7817 | need_clip (bool): Whether the parameter gradient need to be clipped |
| 7818 | in optimizer. Default is True. |
| 7819 | """ |
| 7820 | |
| 7821 | @dygraph_only |
| 7822 | def __init__(self, *args, **kwargs): |
| 7823 | if (len(args) > 0 and isinstance(args[0], list)) or 'shape' in kwargs: |
| 7824 | self.__init_by_shape__(*args, **kwargs) |
| 7825 | else: |
| 7826 | self.__init_by_tensor__(*args, **kwargs) |
| 7827 | |
| 7828 | def __init_by_shape__(self, shape, dtype, **kwargs): |
| 7829 | if shape is None: |
| 7830 | raise ValueError("The shape of Parameter should not be None") |
| 7831 | if dtype is None: |
| 7832 | raise ValueError("The dtype of Parameter should not be None") |
| 7833 | |
| 7834 | for each in shape: |
| 7835 | if each < 0: |
| 7836 | raise ValueError( |
| 7837 | f"Each dimension of shape for Parameter must be greater than 0, but received {list(shape)}" |
| 7838 | ) |
| 7839 | |
| 7840 | if dtype is not None: |
| 7841 | dtype = convert_to_proto_type(dtype) |
| 7842 | else: |
| 7843 | dtype = core.VarDesc.VarType.FP32 |
| 7844 | |
| 7845 | name = kwargs.get("name", unique_name.generate("_eager_param_base")) |
| 7846 | |
| 7847 | if isinstance(shape, core.eager.Tensor): |
| 7848 | shape = shape.numpy() |
| 7849 | |
| 7850 | super().__init__( |
| 7851 | dtype, |
| 7852 | list(shape) if shape else [], |
| 7853 | name, |
no outgoing calls