Make web requests in parallel with custom options for each request. Yield responses as they come in. Similar to `request_batch` except it allows individual arguments for each URL. Args: urls_and_kwargs (list[tuple]): List of tuples in the format: (url, kwargs,
(self, urls_and_kwargs)
| 152 | break |
| 153 | |
| 154 | async def request_custom_batch(self, urls_and_kwargs): |
| 155 | """ |
| 156 | Make web requests in parallel with custom options for each request. Yield responses as they come in. |
| 157 | |
| 158 | Similar to `request_batch` except it allows individual arguments for each URL. |
| 159 | |
| 160 | Args: |
| 161 | urls_and_kwargs (list[tuple]): List of tuples in the format: (url, kwargs, custom_tracker) |
| 162 | where custom_tracker is an optional value for your own internal use. You may use it to |
| 163 | help correlate requests, etc. |
| 164 | |
| 165 | Examples: |
| 166 | >>> urls_and_kwargs = [ |
| 167 | >>> ("http://evilcorp.com/1", {"method": "GET"}, "request-1"), |
| 168 | >>> ("http://evilcorp.com/2", {"method": "POST"}, "request-2"), |
| 169 | >>> ] |
| 170 | >>> async for url, kwargs, custom_tracker, response in self.helpers.request_custom_batch( |
| 171 | >>> urls_and_kwargs |
| 172 | >>> ): |
| 173 | >>> if response is not None and response.status_code == 200: |
| 174 | >>> self.hugesuccess(response) |
| 175 | """ |
| 176 | agen = self.run_and_yield("request_custom_batch", urls_and_kwargs) |
| 177 | while 1: |
| 178 | try: |
| 179 | yield await agen.__anext__() |
| 180 | except (StopAsyncIteration, GeneratorExit): |
| 181 | await agen.aclose() |
| 182 | break |
| 183 | |
| 184 | async def download(self, url, **kwargs): |
| 185 | """ |