Class CachedReader

caffe2/python/cached_reader.py:16–133

class CachedReader(DBFileReader):

    default_name_suffix = 'cached_reader'

    """Reader with persistent in-file cache.

    Example usage:
    cached_reader = CachedReader(
        reader,
        db_path='/tmp/cache.db',
        db_type='LevelDB',
    )
    build_cache_step = cached_reader.build_cache_step()
    with LocalSession() as session:
        session.run(build_cache_step)

    Every time new CachedReader is created, it's expected that
    db_path exists before calling .setup_ex(...) and .read(...).

    If db_path doesn't exist, it's expected build_cache_step to be called
    first to build a cache at db_path.

    build_cache_step will check existence of provided db_path and in case
    it's missing will initialize it by reading data from original reader.
    All consequent attempts to read will ignore original reader
    (i.e. no additional data will be read from it).

    Args:
        original_reader: Reader.
            If provided, it's the original reader used to build the cache file.
        db_path: str.

    Optional Args:
        db_type: str. DB type of file. A db_type is registed by
            `REGISTER_CAFFE2_DB(<db_type>, <DB Class>)`.
            Default to 'LevelDB'.
        name: str or None. Name of CachedReader.
            Optional name to prepend to blobs that will store the data.
            Default to '<db_name>_<default_name_suffix>'.
        batch_size: int.
            How many examples are read for each time the read_net is run.
            Defaults to 100.
        loop_over: bool.
            If True given, will go through examples in random order endlessly.
            Defaults to False.
    """
    def __init__(
        self,
        original_reader,
        db_path,
        db_type='LevelDB',
        name=None,
        batch_size=100,
        loop_over=False,
    ):
        assert original_reader is not None, "original_reader can't be None"
        self.original_reader = original_reader