Fetches a `tfds.core.DatasetBuilder` by string name. Args: name: `str`, the registered name of the `DatasetBuilder` (the class name as camel or snake case: `MyDataset` or `my_dataset`). This can be either `'dataset_name'` or `'dataset_name/config_name'` for datasets with `Bu
(
name: str,
*,
try_gcs: bool = False,
**builder_kwargs: Any,
)
| 129 | @error_utils.reraise_with_context(registered.DatasetNotFoundError) |
| 130 | @tfds_logging.builder() |
| 131 | def builder( |
| 132 | name: str, |
| 133 | *, |
| 134 | try_gcs: bool = False, |
| 135 | **builder_kwargs: Any, |
| 136 | ) -> dataset_builder.DatasetBuilder: |
| 137 | """Fetches a `tfds.core.DatasetBuilder` by string name. |
| 138 | |
| 139 | Args: |
| 140 | name: `str`, the registered name of the `DatasetBuilder` (the class name as |
| 141 | camel or snake case: `MyDataset` or `my_dataset`). This can be either |
| 142 | `'dataset_name'` or `'dataset_name/config_name'` for datasets with |
| 143 | `BuilderConfig`s. As a convenience, this string may contain |
| 144 | comma-separated keyword arguments for the builder. For example |
| 145 | `'foo_bar/a=True,b=3'` would use the `FooBar` dataset passing the keyword |
| 146 | arguments `a=True` and `b=3` (for builders with configs, it would be |
| 147 | `'foo_bar/zoo/a=True,b=3'` to use the `'zoo'` config and pass to the |
| 148 | builder keyword arguments `a=True` and `b=3`). |
| 149 | try_gcs: `bool`, if True, `tfds.load` will see if the dataset exists on the |
| 150 | public GCS bucket before building it locally. This is equivalent to |
| 151 | passing `data_dir='gs://tfds-data/datasets'`. Warning: `try_gcs` is |
| 152 | different than `builder_kwargs.download_config.try_download_gcs`. |
| 153 | `try_gcs` (default: False) overrides `data_dir` to be the public GCS |
| 154 | bucket. `try_download_gcs` (default: True) allows downloading from GCS |
| 155 | while keeping a different `data_dir` than the public GCS bucket. So, to |
| 156 | fully bypass GCS, please use `try_gcs=False` and |
| 157 | `download_and_prepare_kwargs={'download_config': |
| 158 | tfds.core.download.DownloadConfig(try_download_gcs=False)})`. |
| 159 | **builder_kwargs: `dict` of keyword arguments passed to the |
| 160 | `tfds.core.DatasetBuilder`. |
| 161 | |
| 162 | Returns: |
| 163 | A `tfds.core.DatasetBuilder`. |
| 164 | |
| 165 | Raises: |
| 166 | DatasetNotFoundError: if `name` is unrecognized. |
| 167 | """ |
| 168 | # 'kaggle:my_ds/config:1.0.0' -> ( |
| 169 | # DatasetName('kaggle:my_ds'), {'version': '1.0.0', 'config': 'conf0'} |
| 170 | # ) |
| 171 | name, builder_kwargs = naming.parse_builder_name_kwargs( |
| 172 | name, **builder_kwargs |
| 173 | ) |
| 174 | |
| 175 | def get_dataset_repr() -> str: |
| 176 | return f'dataset "{name}", builder_kwargs "{builder_kwargs}"' |
| 177 | |
| 178 | # `try_gcs` currently only supports non-community datasets |
| 179 | if try_gcs and not name.namespace and gcs_utils.is_dataset_on_gcs(str(name)): |
| 180 | data_dir = builder_kwargs.get('data_dir') |
| 181 | if data_dir: |
| 182 | raise ValueError( |
| 183 | f'Cannot have both `try_gcs=True` and `data_dir={data_dir}`' |
| 184 | f' explicitly set. Wrong arguments for {get_dataset_repr()}' |
| 185 | ) |
| 186 | builder_kwargs['data_dir'] = gcs_utils.gcs_path('datasets') |
| 187 | if name.namespace: |
| 188 | if name.namespace == 'huggingface': |
no test coverage detected