Function
extend_embedding_weights
(state_weights, model_weights)
Source from the content-addressed store, hash-verified
| 32 | |
| 33 | # Model. |
| 34 | def extend_embedding_weights(state_weights, model_weights): |
| 35 | original_length = state_weights.shape[0] |
| 36 | assert original_length <= args.max_position_embeddings + 1 |
| 37 | new_weights = model_weights.clone() |
| 38 | new_weights[:original_length] = state_weights |
| 39 | return new_weights |
| 40 | |
| 41 | if args.block_lm: |
| 42 | if "transformer.block_position_embeddings.weight" in sd["module"]: |
Tested by
no test coverage detected