MCPcopy Create free account
hub / github.com/apify/crawlee-python / get_data

Method get_data

src/crawlee/crawlers/_basic/_basic_crawler.py:868–896  ·  view source on GitHub ↗

Retrieve data from a `Dataset`. This helper method simplifies the process of retrieving data from a `Dataset`. It opens the specified one and then retrieves the data based on the provided parameters. Args: dataset_id: The ID of the `Dataset`. dataset

(
        self,
        dataset_id: str | None = None,
        dataset_name: str | None = None,
        dataset_alias: str | None = None,
        **kwargs: Unpack[GetDataKwargs],
    )

Source from the content-addressed store, hash-verified

866 await store.persist_autosaved_values()
867
868 async def get_data(
869 self,
870 dataset_id: str | None = None,
871 dataset_name: str | None = None,
872 dataset_alias: str | None = None,
873 **kwargs: Unpack[GetDataKwargs],
874 ) -> DatasetItemsListPage:
875 """Retrieve data from a `Dataset`.
876
877 This helper method simplifies the process of retrieving data from a `Dataset`. It opens the specified
878 one and then retrieves the data based on the provided parameters.
879
880 Args:
881 dataset_id: The ID of the `Dataset`.
882 dataset_name: The name of the `Dataset` (global scope, named storage).
883 dataset_alias: The alias of the `Dataset` (run scope, unnamed storage).
884 kwargs: Keyword arguments to be passed to the `Dataset.get_data()` method.
885
886 Returns:
887 The retrieved data.
888 """
889 dataset = await Dataset.open(
890 id=dataset_id,
891 name=dataset_name,
892 alias=dataset_alias,
893 storage_client=self._service_locator.get_storage_client(),
894 configuration=self._service_locator.get_configuration(),
895 )
896 return await dataset.get_data(**kwargs)
897
898 async def export_data(
899 self,

Calls 3

get_storage_clientMethod · 0.80
get_configurationMethod · 0.80
openMethod · 0.45