Function data_source

tensorflow_datasets/core/load.py:724–840

Gets a data source from the named dataset. `tfds.data_source` is a convenience method that: 1. Fetches the `tfds.core.DatasetBuilder` by name: ```python builder = tfds.builder(name, data_dir=data_dir, **builder_kwargs) ``` 2. Generates the data (when `download=True`):

(
    name: str,
    *,
    split: Tree[splits_lib.SplitArg] | None = None,
    data_dir: epath.PathLike | None = None,
    download: bool = True,
    decoders: TreeDict[decode.partial_decode.DecoderArg] | None = None,
    deserialize_method: decode.DeserializeMethod = decode.DeserializeMethod.DESERIALIZE_AND_DECODE,
    builder_kwargs: dict[str, Any] | None = None,
    download_and_prepare_kwargs: dict[str, Any] | None = None,
    try_gcs: bool = False,
)

@tfds_logging.data_source()
def data_source(
    name: str,
    *,
    split: Tree[splits_lib.SplitArg] | None = None,
    data_dir: epath.PathLike | None = None,
    download: bool = True,
    decoders: TreeDict[decode.partial_decode.DecoderArg] | None = None,
    deserialize_method: decode.DeserializeMethod = decode.DeserializeMethod.DESERIALIZE_AND_DECODE,
    builder_kwargs: dict[str, Any] | None = None,
    download_and_prepare_kwargs: dict[str, Any] | None = None,
    try_gcs: bool = False,
) -> type_utils.ListOrTreeOrElem[Sequence[Any]]:
  """Gets a data source from the named dataset.

  `tfds.data_source` is a convenience method that:

  1. Fetches the `tfds.core.DatasetBuilder` by name:

     ```python
     builder = tfds.builder(name, data_dir=data_dir, **builder_kwargs)
     ```

  2. Generates the data (when `download=True`):

     ```python
     builder.download_and_prepare(**download_and_prepare_kwargs)
     ```

  3. Gets the data source:

     ```python
     ds = builder.as_data_source(split=split)
     ```

  You can consume data sources:

  - In Python by iterating over them:

  ```python
  for example in ds['train']:
    print(example)
  ```

  - With a DataLoader (e.g., with
  [Pytorch](https://pytorch.org/docs/stable/data.html)).

  **Warning**: calling this function might potentially trigger the download
  of hundreds of GiB to disk. Refer to the `download` argument.

  Args:
    name: `str`, the registered name of the `DatasetBuilder` (the snake case
      version of the class name). The config and version can also be specified
      in the name as follows: `'dataset_name[/config_name][:version]'`. For
      example, `'movielens/25m-ratings'` (for the latest version of
      `'25m-ratings'`), `'movielens:0.1.0'` (for the default config and version
      0.1.0), or`'movielens/25m-ratings:0.1.0'`. Note that only the latest
      version can be generated, but old versions can be read if they are present
      on disk. For convenience, the `name` parameter can contain comma-separated