Remove fulltext files that are catalogs/lists rather than real content.

1. Remove image links that appear within the very first content block (often document cover page artifacts / version tables mistakenly par

Replace 3+ consecutive blank lines with 2.

Stage and commit a list of files.

Convert a single PDF and return the Markdown text.

Create and checkout a new git branch.

Create a GitHub PR and return the PR URL.

Try to match extracted content against any index entry without fulltext. Returns the best matching entry, or None.

docling often emits the document title/date 2-3 times at the top. Remove exact duplicate consecutive paragraphs (non-table, non-heading).

Fix tables where docling expanded merged cells by repeating content across columns. Strategy: process row by row within each table. - If

Derive unique L1 codes from a list of L2/L3 codes.

Derive a local file path for an image URL. Uses the URL path structure to organize images by category.

Try to discover the PDF download URL from an FDA guidance page.

Phase 1: Search for doc/docx URLs for entries without fulltext.

Convert DOCX to Markdown via mammoth (DOCX -> HTML) + html2text (HTML -> MD). Images are compressed and embedded as base64 JPEG data URIs.

Phase 2: Download doc/docx files, validate, extract, cross-match. For each entry, tries ALL candidate URLs. For each URL: 1. Download the fil

Download a single image. Returns True on success.

Append full-text section to .zh.md if not already present. Returns True if modified.

Extract ALL doc/docx download URLs from search results (deduplicated).

Extract classification codes embedded in the slug/filename.

Extract classification codes from text using keyword context and broad matching. Returns (high_confidence_codes, medium_confidence_codes)

Extract YAML front matter from Markdown file.

Extract YAML frontmatter and body from markdown text.

Extract embedded images from a docx and save to IMAGES_DIR/{slug}/. Returns a mapping from relationship ID (rId) to the markdown image path,

Returns (content_type, data) or None.

Paginate the full standards list (general + professional domains). NIFDC ``qxqwk.do`` lists only 228 general-domain standards by default. To

Step 1: Fetch from EUR-Lex and parse standards list.

Scrape the IMDRF document page to find DOCX and PDF download links.

Fetch PDF using curl-cffi with Chrome TLS impersonation.

Ensure we have .docx bytes. Converts .doc via LibreOffice if needed.

1. Re-level flat headings: flat '## 1)' -> '### 1)' 2. Convert fake headings (like '## OF CLASS I MEDICAL DEVICES') to normal text.

Cleans up docling's list bullets that include original PDF bullets like '•', '-', or 'o'. Converts '-  Text', '- - Text', etc. into '- Text'

Fix all broken tables in markdown content.

Generate .en.md file alongside .zh.md. Returns True if created.

Generate guidance pages for a framework. Returns list of changed files.

Generate guidance index page (ZH) for a framework.

Generate a stable item ID from framework + title keywords.

Step 3: Generate category-split JSON files for data layer.

Generate a markdown table from standards list.

Generate or update a docs/zh/ index page for a section.

Generate VitePress pages for _shared/ sections (iso_iec, imdrf). Returns list of changed files.

Generate docs/.vitepress/sidebar.json with dynamic sidebar items. VitePress uses longest-prefix matching for sidebar: /zh/nmpa/

Generate all EU MDR standards sub-pages. Returns list of changed files.

Generate a page for one L1 category with all L2/L3 items.

Read all .zh.md files in a data directory and extract metadata.

Determine EN and ZH draft file paths based on update category.

Filter entries that have substantive English .en.md content. Returns a list of entries with English title available.

Extract English title from YAML front matter.

Get the output path for a regulation fulltext.

Convert HTML content to clean Markdown.

Return True if this table block is a table-of-contents.

Return True if a Markdown table row looks like a TOC entry (dot-filled).

Joins paragraphs that were prematurely split across lines (often due to PDF page/column breaks). Looks for paragraphs ending without terminal

List all posts and their detected subcategories.

Fetch all docs and show how they would be mapped.

Load and merge all news items from JSON files.

Load all standards from JSON files with their category.

Load standards from docmcp standards_2025.json.

Load document entries from _index.json files.

Load all standards from knowledge repo category files.

Load regulatory overview content for a framework if available.

Load regulation entries from _index.json.

Load source URLs from .zh.md front matter.

Load valid L2 and L3 codes from classification catalog. Returns (valid_l2, valid_l3, l3_to_l1_map)