Method update

numpy_ml/rl_models/agents.py:1600–1618

Update the priority queue with the most recent (state, action) pair and perform random-sample one-step tabular Q-planning. Notes ----- The planning algorithm uses a priority queue to retrieve the state-action pairs from the agent's history which will

(self)

        return out

    def update(self):
        """
        Update the priority queue with the most recent (state, action) pair and
        perform random-sample one-step tabular Q-planning.

        Notes
        -----
        The planning algorithm uses a priority queue to retrieve the
        state-action pairs from the agent's history which will result in the
        largest change to its `Q`-value if backed up. When the first pair in
        the queue is backed up, the effect on each of its predecessor pairs is
        computed. If the predecessor's priority is greater than a small
        threshold the pair is added to the queue and the process is repeated
        until either the queue is empty or we exceed `n_simulated_actions`
        updates.
        """
        s, a = self.episode_history["state_actions"][-1]
        self._update_queue(s, a)
        self._simulate_behavior()

    def _update_queue(self, s, a):
        """