Instead of actual data we may store the HDF5 file name and dataset name within the HDF5 file. When access is needed, then data may be loaded from file. Typically this will be used for keeping reference to large dataset that will be loaded in 'lazy' way during processing. Parame
| 164 | |
| 165 | |
| 166 | class RawHDF5Dataset: |
| 167 | """ |
| 168 | Instead of actual data we may store the HDF5 file name and dataset name within the |
| 169 | HDF5 file. When access is needed, then data may be loaded from file. Typically |
| 170 | this will be used for keeping reference to large dataset that will be loaded in |
| 171 | 'lazy' way during processing. |
| 172 | |
| 173 | Parameters |
| 174 | ---------- |
| 175 | abs_path: str |
| 176 | absolute path to the HDF5 file |
| 177 | dset_name: str |
| 178 | name of the dataset in the HDF5 file |
| 179 | shape: tuple |
| 180 | the object is expected to have additional attribute `shape`. Keeping valid shape |
| 181 | information is very convenient. There is no check, so `shape` may be any value, but |
| 182 | typically this should be a tuple with actual dataset shape. |
| 183 | """ |
| 184 | |
| 185 | def __init__(self, _abs_path, _dset_name, shape): |
| 186 | self.abs_path = os.path.abspath(os.path.expanduser(_abs_path)) |
| 187 | self.dset_name = _dset_name |
| 188 | self.shape = shape |
| 189 | |
| 190 | |
| 191 | def _compute_optimal_chunk_size(chunk_pixels, data_chunksize, data_shape, n_chunks_min=4): |
no outgoing calls