Get perplexity scores given a list of inputs. Args: inputs (List[str]): A list of strings. mask_length (Optional[List[int]]): A list of mask lengths. If provided, the perplexity scores will be calculated with the first mask_length[i] t
(self,
inputs: List[str],
mask_length: Optional[List[int]] = None)
| 66 | |
| 67 | @abstractmethod |
| 68 | def get_ppl(self, |
| 69 | inputs: List[str], |
| 70 | mask_length: Optional[List[int]] = None) -> List[float]: |
| 71 | """Get perplexity scores given a list of inputs. |
| 72 | |
| 73 | Args: |
| 74 | inputs (List[str]): A list of strings. |
| 75 | mask_length (Optional[List[int]]): A list of mask lengths. If |
| 76 | provided, the perplexity scores will be calculated with the |
| 77 | first mask_length[i] tokens masked out. It's okay to skip |
| 78 | its implementation if advanced features in PPLInfernecer is |
| 79 | not needed. |
| 80 | |
| 81 | Returns: |
| 82 | List[float]: A list of perplexity scores. |
| 83 | """ |
| 84 | raise NotImplementedError(f'{self.__class__.__name__} does not support' |
| 85 | ' ppl-based evaluation yet, try gen-based ' |
| 86 | 'instead.') |
| 87 | |
| 88 | @abstractmethod |
| 89 | def get_ppl_tokenwise( |
no outgoing calls
no test coverage detected