Move rule (atari-reset ResetManager.proc_infos): forward-cumsum the per-index success rates from index 0; the new max starting point is the FIRST index where the cumulative mass reaches move_threshold*window — i.e. march back as far as the practiced success band supports, no
(self, envs)
| 206 | self.success[min(starting_point, self.n)] = float(success) |
| 207 | |
| 208 | def update(self, envs): |
| 209 | """Move rule (atari-reset ResetManager.proc_infos): forward-cumsum the |
| 210 | per-index success rates from index 0; the new max starting point is the |
| 211 | FIRST index where the cumulative mass reaches move_threshold*window — |
| 212 | i.e. march back as far as the practiced success band supports, no |
| 213 | further. If the mass is never reached (success collapsed), nudge the |
| 214 | curriculum forward (easier) by `nudge`.""" |
| 215 | tail = self.success[: self.max_starting_point + 1] |
| 216 | csum = np.cumsum(tail) # forward: mass accumulated up to each index |
| 217 | hits = np.argwhere(csum >= self.move_threshold * self.window) |
| 218 | if len(hits): |
| 219 | new_max = int(hits[0][0]) # earliest index reaching the mass |
| 220 | self.max_starting_point = max(min(new_max, self.max_starting_point), 0) |
| 221 | else: |
| 222 | self.max_starting_point = min(self.max_starting_point + self.nudge, self.max_max) |
| 223 | self.assign(envs) |
| 224 | return self.max_starting_point |
| 225 | |
| 226 | |
| 227 | def load_demo(path): |