Wrapper class for an http response body. This provides a few additional conveniences that do not exist in the urllib3 model: * Set the timeout on the socket (i.e read() timeouts) * Auto validation of content length, if the amount of bytes we read does not match th
| 32 | |
| 33 | |
| 34 | class StreamingBody: |
| 35 | """Wrapper class for an http response body. |
| 36 | |
| 37 | This provides a few additional conveniences that do not exist |
| 38 | in the urllib3 model: |
| 39 | |
| 40 | * Set the timeout on the socket (i.e read() timeouts) |
| 41 | * Auto validation of content length, if the amount of bytes |
| 42 | we read does not match the content length, an exception |
| 43 | is raised. |
| 44 | |
| 45 | """ |
| 46 | |
| 47 | _DEFAULT_CHUNK_SIZE = 1024 |
| 48 | |
| 49 | def __init__(self, raw_stream, content_length): |
| 50 | self._raw_stream = raw_stream |
| 51 | self._content_length = content_length |
| 52 | self._amount_read = 0 |
| 53 | |
| 54 | def set_socket_timeout(self, timeout): |
| 55 | """Set the timeout seconds on the socket.""" |
| 56 | # The problem we're trying to solve is to prevent .read() calls from |
| 57 | # hanging. This can happen in rare cases. What we'd like to ideally |
| 58 | # do is set a timeout on the .read() call so that callers can retry |
| 59 | # the request. |
| 60 | # Unfortunately, this isn't currently possible in requests. |
| 61 | # See: https://github.com/kennethreitz/requests/issues/1803 |
| 62 | # So what we're going to do is reach into the guts of the stream and |
| 63 | # grab the socket object, which we can set the timeout on. We're |
| 64 | # putting in a check here so in case this interface goes away, we'll |
| 65 | # know. |
| 66 | try: |
| 67 | # To further complicate things, the way to grab the |
| 68 | # underlying socket object from an HTTPResponse is different |
| 69 | # in py2 and py3. So this code has been pushed to botocore.compat. |
| 70 | set_socket_timeout(self._raw_stream, timeout) |
| 71 | except AttributeError: |
| 72 | logger.error( |
| 73 | "Cannot access the socket object of " |
| 74 | "a streaming response. It's possible " |
| 75 | "the interface has changed.", |
| 76 | exc_info=True, |
| 77 | ) |
| 78 | raise |
| 79 | |
| 80 | def read(self, amt=None): |
| 81 | """Read at most amt bytes from the stream. |
| 82 | |
| 83 | If the amt argument is omitted, read all data. |
| 84 | """ |
| 85 | try: |
| 86 | chunk = self._raw_stream.read(amt) |
| 87 | except URLLib3ReadTimeoutError as e: |
| 88 | # TODO: the url will be None as urllib3 isn't setting it yet |
| 89 | raise ReadTimeoutError(endpoint_url=e.url, error=e) |
| 90 | self._amount_read += len(chunk) |
| 91 | if amt is None or (not chunk and amt > 0): |
no outgoing calls