Inverse function of sigmoid. Args: x (Tensor): The tensor to do the inverse. eps (float): EPS avoid numerical overflow. Defaults 1e-5. Returns: Tensor: The x has passed the inverse function of sigmoid, has same shape wi
(x, eps=1e-5)
| 38 | |
| 39 | |
| 40 | def inverse_sigmoid(x, eps=1e-5): |
| 41 | """Inverse function of sigmoid. |
| 42 | |
| 43 | Args: |
| 44 | x (Tensor): The tensor to do the |
| 45 | inverse. |
| 46 | eps (float): EPS avoid numerical |
| 47 | overflow. Defaults 1e-5. |
| 48 | Returns: |
| 49 | Tensor: The x has passed the inverse |
| 50 | function of sigmoid, has same |
| 51 | shape with input. |
| 52 | """ |
| 53 | x = x.clamp(min=0, max=1) |
| 54 | x1 = x.clamp(min=eps) |
| 55 | x2 = (1 - x).clamp(min=eps) |
| 56 | return torch.log(x1 / x2) |
| 57 | |
| 58 | |
| 59 | @TRANSFORMER_LAYER.register_module() |