MCPcopy Create free account
hub / github.com/Vchitect/Latte / persistent_class

Function persistent_class

tools/torch_utils/persistence.py:35–130  ·  view source on GitHub ↗

r"""Class decorator that extends a given class to save its source code when pickled. Example: from src.torch_utils import persistence @persistence.persistent_class class MyNetwork(torch.nn.Module): def __init__(self, num_inputs, num_outputs):

(orig_class)

Source from the content-addressed store, hash-verified

33#----------------------------------------------------------------------------
34
35def persistent_class(orig_class):
36 r"""Class decorator that extends a given class to save its source code
37 when pickled.
38
39 Example:
40
41 from src.torch_utils import persistence
42
43 @persistence.persistent_class
44 class MyNetwork(torch.nn.Module):
45 def __init__(self, num_inputs, num_outputs):
46 super().__init__()
47 self.fc = MyLayer(num_inputs, num_outputs)
48 ...
49
50 @persistence.persistent_class
51 class MyLayer(torch.nn.Module):
52 ...
53
54 When pickled, any instance of `MyNetwork` and `MyLayer` will save its
55 source code alongside other internal state (e.g., parameters, buffers,
56 and submodules). This way, any previously exported pickle will remain
57 usable even if the class definitions have been modified or are no
58 longer available.
59
60 The decorator saves the source code of the entire Python module
61 containing the decorated class. It does *not* save the source code of
62 any imported modules. Thus, the imported modules must be available
63 during unpickling, also including `torch_utils.persistence` itself.
64
65 It is ok to call functions defined in the same module from the
66 decorated class. However, if the decorated class depends on other
67 classes defined in the same module, they must be decorated as well.
68 This is illustrated in the above example in the case of `MyLayer`.
69
70 It is also possible to employ the decorator just-in-time before
71 calling the constructor. For example:
72
73 cls = MyLayer
74 if want_to_make_it_persistent:
75 cls = persistence.persistent_class(cls)
76 layer = cls(num_inputs, num_outputs)
77
78 As an additional feature, the decorator also keeps track of the
79 arguments that were used to construct each instance of the decorated
80 class. The arguments can be queried via `obj.init_args` and
81 `obj.init_kwargs`, and they are automatically pickled alongside other
82 object state. A typical use case is to first unpickle a previous
83 instance of a persistent class, and then upgrade it to use the latest
84 version of the source code:
85
86 with open('old_pickle.pkl', 'rb') as f:
87 old_net = pickle.load(f)
88 new_net = MyNetwork(*old_obj.init_args, **old_obj.init_kwargs)
89 misc.copy_params_and_buffers(old_net, new_net, require_all=True)
90 """
91 assert isinstance(orig_class, type)
92 if is_persistent(orig_class):

Callers 1

Calls 2

is_persistentFunction · 0.85
_module_to_srcFunction · 0.85

Tested by

no test coverage detected