This class implemements a client for executing the same queries across multiple OPTIMADE APIs simultaneously, paging and caching the results. By default, all registered OPTIMADE providers will be queried simulateneously and asynchronously, with the results collected into the `al
| 47 | |
| 48 | |
| 49 | class OptimadeClient: |
| 50 | """This class implemements a client for executing the same queries |
| 51 | across multiple OPTIMADE APIs simultaneously, paging and caching the |
| 52 | results. |
| 53 | |
| 54 | By default, all registered OPTIMADE providers will be queried |
| 55 | simulateneously and asynchronously, with the results collected |
| 56 | into the `all_results` attribute, keyed by endpoint, filter |
| 57 | and provider. |
| 58 | |
| 59 | """ |
| 60 | |
| 61 | base_urls: str | Iterable[str] |
| 62 | """A list (or any iterable) of OPTIMADE base URLs to query.""" |
| 63 | |
| 64 | all_results: dict[str, dict[str, dict[str, QueryResults]]] = defaultdict(dict) |
| 65 | """A nested dictionary keyed by endpoint and OPTIMADE filter string that contains |
| 66 | the results from each base URL for that particular filter. |
| 67 | """ |
| 68 | |
| 69 | count_results: dict[str, dict[str, dict[str, int | None]]] = defaultdict(dict) |
| 70 | """A nested dictionary keyed by endpoint and OPTIMADE filter string that contains |
| 71 | the number of results from each base URL for that particular filter. |
| 72 | """ |
| 73 | |
| 74 | max_results_per_provider: int | None = None |
| 75 | """Maximum number of results to downlod per provider. If None, will |
| 76 | download all. |
| 77 | """ |
| 78 | |
| 79 | property_lists: dict[str, dict[str, list[str]]] = defaultdict(dict) |
| 80 | """A dictionary containing list of properties served by each database, |
| 81 | broken down by entry type, then database. |
| 82 | """ |
| 83 | |
| 84 | headers: dict = {"User-Agent": f"optimade-python-tools/{__version__}"} |
| 85 | """Additional HTTP headers.""" |
| 86 | |
| 87 | http_timeout: httpx.Timeout = httpx.Timeout(10.0, read=1000.0) |
| 88 | """The timeout to use for each HTTP request.""" |
| 89 | |
| 90 | max_attempts: int |
| 91 | """The maximum number of times to repeat a failed query before giving up.""" |
| 92 | |
| 93 | max_requests_per_provider: int = 2_000_000 |
| 94 | """An upper limit guard rail to avoid infinite hanging of the client on malformed APIs. |
| 95 | If available, a better value will be estimated for each API based on the total number of entries. |
| 96 | |
| 97 | """ |
| 98 | |
| 99 | use_async: bool |
| 100 | """Whether or not to make all requests asynchronously using asyncio.""" |
| 101 | |
| 102 | callbacks: list[Callable[[str, dict], None | dict]] | None = None |
| 103 | """A list of callbacks to execute after each successful request, used |
| 104 | to e.g., write to a file, add results to a database or perform additional |
| 105 | filtering. |
| 106 |