Class StatMonitorParamSetter

tensorpack/callbacks/param.py:362–443

Change the param by monitoring the change of a scalar statistics. The param will be changed when the scalar does not decrease/increase enough. Once triggered, this callback observes the latest **one** value of ``stat_name``, from the monitor backend. This callback will then change

class StatMonitorParamSetter(HyperParamSetter):
    """
    Change the param by monitoring the change of a scalar statistics.
    The param will be changed when the scalar does not decrease/increase enough.

    Once triggered, this callback observes the latest **one** value of ``stat_name``, from the monitor backend.

    This callback will then change a hyperparameter ``param`` by ``new_value = value_func(old_value)``, if:
    ``min(history) >= history[0] - threshold``, where
    ``history = [the most recent k observations of stat_name]``

    Note:
        The statistics of interest must be created at a frequency higher than or equal to this callback.
        For example, using ``PeriodicTrigger(StatMonitorParamSetter(...), every_k_steps=100)``
        is meaningless if the statistics to be monitored is only updated every 500 steps.

        Callbacks are executed in order. Therefore, if the statistics to be monitored
        is created after this callback, the behavior of this callback may get delayed.

    Example:

        If validation error wasn't decreasing for 5 epochs, decay the learning rate by 0.2:

        .. code-block:: python

            StatMonitorParamSetter('learning_rate', 'val-error',
                                    lambda x: x * 0.2, threshold=0, last_k=5)
    """

    _enable_before_train = False

    def __init__(self, param, stat_name, value_func, threshold,
                 last_k, reverse=False):
        """
        Args:
            param: same as in :class:`HyperParamSetter`.
            stat_name (str): name of the statistics.
            value_func (float -> float): a function which returns a new value
                taking the old value.
            threshold (float): change threshold.
            last_k (int): use last k observations of statistics.
            reverse (bool): monitor increasing instead of decreasing.
                If True, ``param`` will be changed when ``max(history) <= history[0] + threshold``.
        """
        super(StatMonitorParamSetter, self).__init__(param)
        self.stat_name = stat_name
        self.value_func = value_func
        self.history = deque(maxlen=last_k)
        self.threshold = threshold
        self.reverse = reverse

    def _get_value_to_set(self):
        try:
            last = self.trainer.monitors.get_history(self.stat_name)[-1]
        except (KeyError, IndexError):
            logger.warn(
                "[StatMonitorParamSetter] No history data available for key '{}'.".format(self.stat_name))
            return None