Divides an iterable into chunks of specified size or based on a given function. Useful for batching Parameters: - iter: The input iterable to be divided into chunks. - n: An integer representing the size of each chunk. Default is 0. - fn: A function
(_iter, n: int = 0, fn=None)
| 476 | |
| 477 | @staticmethod |
| 478 | def get_chunks(_iter, n: int = 0, fn=None): |
| 479 | """ |
| 480 | Divides an iterable into chunks of specified size or based on a given function. |
| 481 | Useful for batching |
| 482 | |
| 483 | Parameters: |
| 484 | - iter: The input iterable to be divided into chunks. |
| 485 | - n: An integer representing the size of each chunk. Default is 0. |
| 486 | - fn: A function that takes the current index and the iterable as arguments and returns the size of the chunk. Default is None. |
| 487 | |
| 488 | Returns: |
| 489 | An iterator that yields chunks of the input iterable. |
| 490 | |
| 491 | Example usage: |
| 492 | ``` |
| 493 | data = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] |
| 494 | for chunk in chunks(data, 3): |
| 495 | print(chunk) |
| 496 | ``` |
| 497 | Output: |
| 498 | ``` |
| 499 | [1, 2, 3] |
| 500 | [4, 5, 6] |
| 501 | [7, 8, 9] |
| 502 | [10] |
| 503 | ``` |
| 504 | """ |
| 505 | arr = [] |
| 506 | _iter = tuple(_iter) |
| 507 | for i, x in enumerate(_iter): |
| 508 | arr.append(x) |
| 509 | if len(arr) == (fn(i, _iter) if fn else n): |
| 510 | yield arr |
| 511 | arr = [] |
| 512 | |
| 513 | if arr: |
| 514 | yield arr |