Class GettableFile

dwave/cloud/upload.py:119–206

Provide thread-safe memory buffer-like random read access to a file-like object via an efficient item getter interface. Args: fp (:class:`io.BufferedIOBase`/:class:`io.RawIOBase`/binary-file-like): A file-like object that supports seek and readinto operations.

class GettableFile(GettableBase):
    """Provide thread-safe memory buffer-like random read access to a file-like
    object via an efficient item getter interface.

    Args:
        fp (:class:`io.BufferedIOBase`/:class:`io.RawIOBase`/binary-file-like):
            A file-like object that supports seek and readinto operations.
            Thread-safety of these operations is not assumed.

        strict (bool, default=True):
            Require file-like object to be a :class:`io.BufferedIOBase` or
            :class:`io.RawIOBase` instance.

    Note:
        :class:`.GettableFile` implements :class:`.Gettable` interface over a
        file-like object that supports buffered binary read access.

    Example:
        Access overlapping segments of a file from multiple threads::

            with open('/path/to/file', 'rb') as fp:   # binary mode, read access
                gf = GettableFile(fp):

                # in thread 1:
                seg = gf[0:10]

                # in thread 2:
                seg = gf[5:15]

    """

    def __init__(self, fp, strict=True):
        if strict:
            valid = lambda f: (
                isinstance(f, (io.BufferedIOBase, io.RawIOBase))
                and f.seekable() and f.readable())
        else:
            valid = lambda f: all([
                hasattr(f, 'readinto'), hasattr(f, 'seek'), hasattr(f, 'tell')])

        if not valid(fp):
            raise TypeError("expected file-like, seekable, readable object")

        # store file size, assuming it won't change
        self._size = fp.seek(0, os.SEEK_END)
        if self._size is None:
            # handle non-python3 and/or non-standard file seek() impl.
            # (like tempfile.SpooledTemporaryFile)
            # note: not thread-safe!
            self._size = fp.tell()

        self._fp = fp

        # multiple threads will be accessing the underlying file 
        self._lock = threading.RLock()

    def __len__(self):
        return self._size