MCPcopy Create free account
hub / github.com/PaddlePaddle/Paddle / _legacy_load

Function _legacy_load

python/paddle/framework/io.py:1380–1429  ·  view source on GitHub ↗
(path, **configs)

Source from the content-addressed store, hash-verified

1378
1379
1380def _legacy_load(path, **configs):
1381 load_result = None
1382 config = _parse_load_config(configs)
1383
1384 if os.path.isfile(path) or _is_memory_buffer(path):
1385 # we think path is file means this file is created by paddle.save
1386 if config.safetensors:
1387 from safetensors.paddle import load_file
1388
1389 load_result = load_file(path)
1390 else:
1391 with _open_file_buffer(path, 'rb') as f:
1392 load_result = safe_load_pickle(f, encoding='latin1')
1393 load_result = _pack_loaded_dict(load_result)
1394 if (
1395 not config.keep_name_table
1396 and "StructuredToParameterName@@" in load_result
1397 ):
1398 del load_result["StructuredToParameterName@@"]
1399 else:
1400 # file prefix and directory are compatible cases
1401 model_path, config = _build_load_path_and_config(path, config)
1402 # check whether model file exists
1403 if config.model_filename is None:
1404 model_filename = '__model__'
1405 else:
1406 model_filename = config.model_filename
1407 model_file_path = os.path.join(model_path, model_filename)
1408
1409 if os.path.exists(model_file_path):
1410 # Load state dict by `jit.save/io.save_inference_model` save format
1411 # NOTE(chenweihang): [ Compatibility of save_inference_model save format ]
1412 # The model saved by `save_inference_model` does not completely correspond to
1413 # the information required by the `state_dict` under the dygraph.
1414 # `save_inference_model` not save structured name, we need to remind
1415 # the user to configure the `use_structured_name` argument when `set_state_dict`
1416 # NOTE(chenweihang): `jit.save` doesn't save optimizer state
1417 load_result = _load_state_dict_from_save_inference_model(
1418 model_path, config
1419 )
1420 else:
1421 # load state dict by `io.save_params/persistables` save format
1422 # TODO(chenweihang): [ Now only supports loading parameters separately ]
1423 # If users save all parameters as one file, the [ variable.name -> variable ]
1424 # mapping info will lost, so users need to give variable list, but users build
1425 # variable list in dygraph mode is difficult, we recommend users to use
1426 # paddle.static.load_program_state in this case
1427 load_result = _load_state_dict_from_save_params(model_path)
1428
1429 return load_result

Callers 1

loadFunction · 0.85

Calls 10

_is_memory_bufferFunction · 0.85
_open_file_bufferFunction · 0.85
safe_load_pickleFunction · 0.85
_pack_loaded_dictFunction · 0.85
existsMethod · 0.80
_parse_load_configFunction · 0.70
joinMethod · 0.45

Tested by

no test coverage detected