MCPcopy Create free account
hub / github.com/OpenDriveLab/ReSim / copy_layer_norm

Function copy_layer_norm

SwissArmyTransformer/examples/gpt2/transform_param.py:74–87  ·  view source on GitHub ↗
(src, dst)

Source from the content-addressed store, hash-verified

72 assert (dst_dic[k].data == src_dic[k].data).all()
73
74def copy_layer_norm(src, dst):
75 src_ln = []
76 for k, v in src.named_parameters():
77 if 'ln' in k.lower():
78 src_ln.append((k, v))
79 dst_ln = []
80 for k, v in dst.named_parameters():
81 if 'layernorm' in k.lower():
82 dst_ln.append((k, v))
83 assert len(src_ln) == len(dst_ln)
84 for kvs, kvd in zip(src_ln, dst_ln):
85 assert kvd[1].data.shape == kvs[1].data.shape
86 kvd[1].data = kvs[1].data
87 assert (kvd[1].data == kvs[1].data).all()
88
89def copy_linear_to_conv1d(src, dst):
90 dst.weight.data = src.weight.data.transpose(0, 1)

Callers 1

transform_weightFunction · 0.70

Calls 1

appendMethod · 0.80

Tested by

no test coverage detected