MCPcopy Create free account
hub / github.com/aboutcode-org/scancode-toolkit / fetch_abouts_and_licenses

Function fetch_abouts_and_licenses

etc/scripts/utils_thirdparty.py:1998–2130  ·  view source on GitHub ↗

Given a thirdparty dir, add missing ABOUT. LICENSE and NOTICE files using best efforts: - use existing ABOUT files - try to load existing remote ABOUT files - derive from existing distribution with same name and latest version that would have such ABOUT file - extract

(dest_dir=THIRDPARTY_DIR, use_cached_index=False)

Source from the content-addressed store, hash-verified

1996
1997
1998def fetch_abouts_and_licenses(dest_dir=THIRDPARTY_DIR, use_cached_index=False):
1999 """
2000 Given a thirdparty dir, add missing ABOUT. LICENSE and NOTICE files using
2001 best efforts:
2002
2003 - use existing ABOUT files
2004 - try to load existing remote ABOUT files
2005 - derive from existing distribution with same name and latest version that
2006 would have such ABOUT file
2007 - extract ABOUT file data from distributions PKGINFO or METADATA files
2008
2009 Use available existing on-disk cached index if use_cached_index is True.
2010 """
2011
2012 def get_other_dists(_package, _dist):
2013 """
2014 Return a list of all the dists from `_package` that are not the `_dist`
2015 object
2016 """
2017 return [d for d in _package.get_distributions() if d != _dist]
2018
2019 local_packages = get_local_packages(directory=dest_dir)
2020 packages_by_name = defaultdict(list)
2021 for local_package in local_packages:
2022 distributions = list(local_package.get_distributions())
2023 distribution = distributions[0]
2024 packages_by_name[distribution.name].append(local_package)
2025
2026 for local_package in local_packages:
2027 for local_dist in local_package.get_distributions():
2028 local_dist.load_about_data(dest_dir=dest_dir)
2029 local_dist.set_checksums(dest_dir=dest_dir)
2030
2031 # if has key data we may look to improve later, but we can move on
2032 if local_dist.has_key_metadata():
2033 local_dist.save_about_and_notice_files(dest_dir=dest_dir)
2034 local_dist.fetch_license_files(dest_dir=dest_dir, use_cached_index=use_cached_index)
2035 continue
2036
2037 # lets try to get from another dist of the same local package
2038 for otherd in get_other_dists(local_package, local_dist):
2039 updated = local_dist.update_from_other_dist(otherd)
2040 if updated and local_dist.has_key_metadata():
2041 break
2042
2043 # if has key data we may look to improve later, but we can move on
2044 if local_dist.has_key_metadata():
2045 local_dist.save_about_and_notice_files(dest_dir=dest_dir)
2046 local_dist.fetch_license_files(dest_dir=dest_dir, use_cached_index=use_cached_index)
2047 continue
2048
2049 # try to get another version of the same package that is not our version
2050 other_local_packages = [
2051 p
2052 for p in packages_by_name[local_package.name]
2053 if p.version != local_package.version
2054 ]
2055 other_local_version = other_local_packages and other_local_packages[-1]

Callers

nothing calls this directly

Calls 14

get_local_packagesFunction · 0.85
get_other_distsFunction · 0.85
get_distributionsMethod · 0.80
load_about_dataMethod · 0.80
set_checksumsMethod · 0.80
has_key_metadataMethod · 0.80
fetch_license_filesMethod · 0.80
get_package_versionsMethod · 0.80
load_pkginfo_dataMethod · 0.80

Tested by

no test coverage detected