Method
__init__
(self, window_length: int, num_sink_tokens: int)
Source from the content-addressed store, hash-verified
| 819 | """ |
| 820 | |
| 821 | def __init__(self, window_length: int, num_sink_tokens: int) -> None: |
| 822 | super().__init__() |
| 823 | self.key_cache: List[torch.Tensor] = [] |
| 824 | self.value_cache: List[torch.Tensor] = [] |
| 825 | self.window_length = window_length |
| 826 | self.num_sink_tokens = num_sink_tokens |
| 827 | self.cos_sin_rerotation_cache = {} |
| 828 | self._cos_cache = None |
| 829 | self._sin_cache = None |
| 830 | self._seen_tokens = 0 # Used in `generate` to keep tally of how many tokens the cache has seen |
| 831 | |
| 832 | @staticmethod |
| 833 | def _rotate_half(x): |
Callers
nothing calls this directly
Tested by
no test coverage detected