Method fetch

qlib/data/dataset/handler.py:198–277

fetch data from underlying data source Design motivation: - providing a unified interface for underlying data. - Potential to make the interface more friendly. - User can improve performance when fetching data in this extra layer Parameters

(
        self,
        selector: Union[pd.Timestamp, slice, str, pd.Index] = slice(None, None),
        level: Union[str, int] = "datetime",
        col_set: Union[str, List[str]] = DataHandlerABC.CS_ALL,
        data_key: DATA_KEY_TYPE = DataHandlerABC.DK_I,
        squeeze: bool = False,
        proc_func: Optional[Callable] = None,
    )

        # TODO: cache

    def fetch(
        self,
        selector: Union[pd.Timestamp, slice, str, pd.Index] = slice(None, None),
        level: Union[str, int] = "datetime",
        col_set: Union[str, List[str]] = DataHandlerABC.CS_ALL,
        data_key: DATA_KEY_TYPE = DataHandlerABC.DK_I,
        squeeze: bool = False,
        proc_func: Optional[Callable] = None,
    ) -> pd.DataFrame:
        """
        fetch data from underlying data source

        Design motivation:
        - providing a unified interface for underlying data.
        - Potential to make the interface more friendly.
        - User can improve performance when fetching data in this extra layer

        Parameters
        ----------
        selector : Union[pd.Timestamp, slice, str]
            describe how to select data by index
            It can be categories as following

            - fetch single index
            - fetch a range of index

                - a slice range
                - pd.Index for specific indexes

            Following conflicts may occur

            - Does ["20200101", "20210101"] mean selecting this slice or these two days?

                - slice have higher priorities

        level : Union[str, int]
            which index level to select the data

        col_set : Union[str, List[str]]

            - if isinstance(col_set, str):

                select a set of meaningful, pd.Index columns.(e.g. features, columns)

                - if col_set == CS_RAW:

                    the raw dataset will be returned.

            - if isinstance(col_set, List[str]):

                select several sets of meaningful columns, the returned data has multiple levels

        proc_func: Callable

            - Give a hook for processing data before fetching
            - An example to explain the necessity of the hook:

                - A Dataset learned some processors to process data which is related to data segmentation