Remove padded sequences from the input. Args: max_len (paddle.Tensor): The maximum length of the input sequences. input_ids (paddle.Tensor): The IDs of the input sequences. seq_lens_this_time (paddle.Tensor): The actual length of each sequence. Returns:
(
max_len: paddle.Tensor,
input_ids: paddle.Tensor,
seq_lens_this_time: paddle.Tensor,
)
| 446 | |
| 447 | |
| 448 | def remove_padding( |
| 449 | max_len: paddle.Tensor, |
| 450 | input_ids: paddle.Tensor, |
| 451 | seq_lens_this_time: paddle.Tensor, |
| 452 | ) -> Tuple[paddle.Tensor, paddle.Tensor, paddle.Tensor, paddle.Tensor, paddle.Tensor]: |
| 453 | """ |
| 454 | Remove padded sequences from the input. |
| 455 | |
| 456 | Args: |
| 457 | max_len (paddle.Tensor): The maximum length of the input sequences. |
| 458 | input_ids (paddle.Tensor): The IDs of the input sequences. |
| 459 | seq_lens_this_time (paddle.Tensor): The actual length of each sequence. |
| 460 | |
| 461 | Returns: |
| 462 | tuple: A tuple containing: |
| 463 | - The sequence IDs with padding removed (paddle.Tensor). |
| 464 | - The padding offsets (paddle.Tensor). |
| 465 | - The cumulative offsets (paddle.Tensor). |
| 466 | - The query sequence lengths (paddle.Tensor). |
| 467 | - The key sequence lengths (paddle.Tensor). |
| 468 | """ |
| 469 | if current_platform.is_cuda(): |
| 470 | cum_offsets_now = paddle.cumsum(max_len - seq_lens_this_time, dtype="int32") |
| 471 | token_num = paddle.sum(seq_lens_this_time) |
| 472 | ( |
| 473 | ids_remove_padding, |
| 474 | cum_offsets, |
| 475 | padding_offset, |
| 476 | cu_seqlens_q, |
| 477 | cu_seqlens_k, |
| 478 | ) = get_padding_offset(input_ids, cum_offsets_now, token_num, seq_lens_this_time) |
| 479 | return ( |
| 480 | ids_remove_padding, |
| 481 | padding_offset, |
| 482 | cum_offsets, |
| 483 | cu_seqlens_q, |
| 484 | cu_seqlens_k, |
| 485 | ) |
| 486 | |
| 487 | |
| 488 | class CpuGuard: |
nothing calls this directly
no test coverage detected