Process the files table to select which content IDs to extract. Adopts the ranking used by manned.org's ``man_pref`` (see www/index.pl) so our pick matches what manned.org and man.archlinux.org serve. For each (name, section) key: 1. Within each package, keep the pkgver with
(
data_dir,
sections,
english_locale_ids,
mans,
packages,
pkg_versions,
matching_sys_ids,
)
| 444 | |
| 445 | |
| 446 | def select_content_ids( |
| 447 | data_dir, |
| 448 | sections, |
| 449 | english_locale_ids, |
| 450 | mans, |
| 451 | packages, |
| 452 | pkg_versions, |
| 453 | matching_sys_ids, |
| 454 | ): |
| 455 | """ |
| 456 | Process the files table to select which content IDs to extract. |
| 457 | |
| 458 | Adopts the ranking used by manned.org's ``man_pref`` (see www/index.pl) |
| 459 | so our pick matches what manned.org and man.archlinux.org serve. For |
| 460 | each (name, section) key: |
| 461 | |
| 462 | 1. Within each package, keep the pkgver with the latest ``released`` |
| 463 | date (tiebreak: lowest shorthash). |
| 464 | 2. If any surviving row is under a standard man-path, drop the rest. |
| 465 | 3. Across packages, pick the row with the latest ``released`` date, |
| 466 | final tiebreak on lowest shorthash. |
| 467 | |
| 468 | Cross-distro and cross-release steps from manned's selector are elided |
| 469 | because upstream filtering (matching_sys_ids) already narrows to one |
| 470 | system. |
| 471 | |
| 472 | Returns: |
| 473 | content_to_manpages: dict mapping content_id -> [(name, section), ...] |
| 474 | """ |
| 475 | # key -> pkg_id -> candidate row dict |
| 476 | # Candidate rows carry everything needed for the multi-step ranking: |
| 477 | # content_id, released, shorthash, is_standard |
| 478 | per_key_per_pkg: dict[tuple[str, str], dict[int, dict]] = {} |
| 479 | |
| 480 | files_path = os.path.join(data_dir, "files.tsv") |
| 481 | count = 0 |
| 482 | skipped_locale = 0 |
| 483 | skipped_section = 0 |
| 484 | skipped_distro = 0 |
| 485 | |
| 486 | for row in parse_tsv(files_path): |
| 487 | # files: pkgver, man, content, shorthash, locale, encoding, filename |
| 488 | pkgver_id = int(row[0]) |
| 489 | man_id = int(row[1]) |
| 490 | content_id = int(row[2]) |
| 491 | try: |
| 492 | shorthash = int(row[3]) |
| 493 | except (ValueError, IndexError): |
| 494 | shorthash = 0 |
| 495 | locale_id = int(row[4]) |
| 496 | filename = row[6] if len(row) > 6 else "" |
| 497 | count += 1 |
| 498 | |
| 499 | if locale_id not in english_locale_ids: |
| 500 | skipped_locale += 1 |
| 501 | continue |
| 502 | |
| 503 | pv = pkg_versions.get(pkgver_id) |
no test coverage detected