Spec to write a final records shard. Attributes: shard_index: Index of the shard. path: The path where to write the shard. index_path: The path where to write index of the records in the corresponding shard. NOTE: Value for this attribute is always set, but usage depends o
| 57 | |
| 58 | @dataclasses.dataclass(frozen=True) |
| 59 | class _ShardSpec: |
| 60 | """Spec to write a final records shard. |
| 61 | |
| 62 | Attributes: |
| 63 | shard_index: Index of the shard. |
| 64 | path: The path where to write the shard. |
| 65 | index_path: The path where to write index of the records in the |
| 66 | corresponding shard. NOTE: Value for this attribute is always set, but |
| 67 | usage depends on whether `write_examples` returned a list of record |
| 68 | positions for each example. |
| 69 | examples_number: Number of examples in shard. |
| 70 | file_instructions: Reading instructions. |
| 71 | """ |
| 72 | |
| 73 | shard_index: int |
| 74 | path: str |
| 75 | index_path: str |
| 76 | examples_number: int |
| 77 | file_instructions: Sequence[shard_utils.FileInstruction] |
| 78 | |
| 79 | |
| 80 | def _raise_error_for_duplicated_keys(example1, example2, example_specs): |