* A dataflow can optionally implement :meth:`__len__`. If not implemented, it will throw :class:`NotImplementedError`. * It returns an integer representing the size of the dataflow. The return value **may not be accurate or meaningful** at all. When sa
(self)
| 84 | """ |
| 85 | |
| 86 | def __len__(self): |
| 87 | """ |
| 88 | * A dataflow can optionally implement :meth:`__len__`. If not implemented, it will |
| 89 | throw :class:`NotImplementedError`. |
| 90 | |
| 91 | * It returns an integer representing the size of the dataflow. |
| 92 | The return value **may not be accurate or meaningful** at all. |
| 93 | When saying the length is "accurate", it means that |
| 94 | :meth:`__iter__` will always yield this many of datapoints before it stops iteration. |
| 95 | |
| 96 | * There could be many reasons why :meth:`__len__` is inaccurate. |
| 97 | For example, some dataflow has dynamic size, if it throws away datapoints on the fly. |
| 98 | Some dataflow mixes the datapoints between consecutive passes over |
| 99 | the dataset, due to parallelism and buffering. |
| 100 | In this case it does not make sense to stop the iteration anywhere. |
| 101 | |
| 102 | * Due to the above reasons, the length is only a rough guidance. |
| 103 | And it's up to the user how to interpret it. |
| 104 | Inside tensorpack it's only used in these places: |
| 105 | |
| 106 | + A default ``steps_per_epoch`` in training, but you probably want to customize |
| 107 | it yourself, especially when using data-parallel trainer. |
| 108 | + The length of progress bar when processing a dataflow. |
| 109 | + Used by :class:`InferenceRunner` to get the number of iterations in inference. |
| 110 | In this case users are **responsible** for making sure that :meth:`__len__` is "accurate". |
| 111 | This is to guarantee that inference is run on a fixed set of images. |
| 112 | |
| 113 | Returns: |
| 114 | int: rough size of this dataflow. |
| 115 | |
| 116 | Raises: |
| 117 | :class:`NotImplementedError` if this DataFlow doesn't have a size. |
| 118 | """ |
| 119 | raise NotImplementedError() |
| 120 | |
| 121 | def reset_state(self): |
| 122 | """ |