Find currently imported submodules used by a function. Submodules used by a function need to be detected and referenced for the function to work correctly at depickling time. Because submodules can be referenced as attribute of their parent package (``package.submodule``), we need a
(code, top_level_dependencies)
| 338 | |
| 339 | |
| 340 | def _find_imported_submodules(code, top_level_dependencies): |
| 341 | """Find currently imported submodules used by a function. |
| 342 | |
| 343 | Submodules used by a function need to be detected and referenced for the |
| 344 | function to work correctly at depickling time. Because submodules can be |
| 345 | referenced as attribute of their parent package (``package.submodule``), we |
| 346 | need a special introspection technique that does not rely on GLOBAL-related |
| 347 | opcodes to find references of them in a code object. |
| 348 | |
| 349 | Example: |
| 350 | ``` |
| 351 | import concurrent.futures |
| 352 | import cloudpickle |
| 353 | def func(): |
| 354 | x = concurrent.futures.ThreadPoolExecutor |
| 355 | if __name__ == '__main__': |
| 356 | cloudpickle.dumps(func) |
| 357 | ``` |
| 358 | The globals extracted by cloudpickle in the function's state include the |
| 359 | concurrent package, but not its submodule (here, concurrent.futures), which |
| 360 | is the module used by func. Find_imported_submodules will detect the usage |
| 361 | of concurrent.futures. Saving this module alongside with func will ensure |
| 362 | that calling func once depickled does not fail due to concurrent.futures |
| 363 | not being imported |
| 364 | """ |
| 365 | |
| 366 | subimports = [] |
| 367 | # check if any known dependency is an imported package |
| 368 | for x in top_level_dependencies: |
| 369 | if ( |
| 370 | isinstance(x, types.ModuleType) |
| 371 | and hasattr(x, "__package__") |
| 372 | and x.__package__ |
| 373 | ): |
| 374 | # check if the package has any currently loaded sub-imports |
| 375 | prefix = x.__name__ + "." |
| 376 | # A concurrent thread could mutate sys.modules, |
| 377 | # make sure we iterate over a copy to avoid exceptions |
| 378 | for name in list(sys.modules): |
| 379 | # Older versions of pytest will add a "None" module to |
| 380 | # sys.modules. |
| 381 | if name is not None and name.startswith(prefix): |
| 382 | # check whether the function can address the sub-module |
| 383 | tokens = set(name[len(prefix) :].split(".")) |
| 384 | if not tokens - set(code.co_names): |
| 385 | subimports.append(sys.modules[name]) |
| 386 | return subimports |
| 387 | |
| 388 | |
| 389 | # relevant opcodes |
no outgoing calls
no test coverage detected