Returns the list of dict(filename, take, skip). This allows for creating your own `tf.data.Dataset` using the low-level TFDS values. Example: ``` file_instructions = info.splits['train[75%:]'].file_instructions instruction_ds = tf.data.Dataset.from_generator( lambd
(self)
| 200 | |
| 201 | @property |
| 202 | def file_instructions(self) -> list[shard_utils.FileInstruction]: |
| 203 | """Returns the list of dict(filename, take, skip). |
| 204 | |
| 205 | This allows for creating your own `tf.data.Dataset` using the low-level |
| 206 | TFDS values. |
| 207 | |
| 208 | Example: |
| 209 | |
| 210 | ``` |
| 211 | file_instructions = info.splits['train[75%:]'].file_instructions |
| 212 | instruction_ds = tf.data.Dataset.from_generator( |
| 213 | lambda: file_instructions, |
| 214 | output_types={ |
| 215 | 'filename': tf.string, |
| 216 | 'take': tf.int64, |
| 217 | 'skip': tf.int64, |
| 218 | }, |
| 219 | ) |
| 220 | ds = instruction_ds.interleave( |
| 221 | lambda f: tf.data.TFRecordDataset( |
| 222 | f['filename']).skip(f['skip']).take(f['take']) |
| 223 | ) |
| 224 | ``` |
| 225 | |
| 226 | When `skip=0` and `take=-1`, the full shard will be read, so the `ds.skip` |
| 227 | and `ds.take` could be skipped. |
| 228 | |
| 229 | Returns: |
| 230 | A `dict(filename, take, skip)` |
| 231 | """ |
| 232 | return _make_file_instructions( |
| 233 | split_infos=[self], |
| 234 | instruction=str(self.name), |
| 235 | ) |
| 236 | |
| 237 | @property |
| 238 | def filenames(self) -> list[str]: |
no test coverage detected