Convert text to URL-safe slug.

Convert a EUR-Lex source URL to the HTML full text URL (fallback).

Split markdown text into translatable and non-translatable segments. Returns list of {"text": str, "translate": bool, "type": str}

Split lines into alternating non-table and table blocks. Returns list of ('text'|'table', [lines]) tuples.

Remove year/revision markers for comparison.

Translate a Chinese markdown document to English.

Pool URLs from all entries sharing the same announcement number. Returns a mapping of entry_id -> set of sibling URLs (excluding spam).

Identify URLs that appear 20+ times across discoveries -- generic/catalog files.

Detect if text has the expected title in header but body content is actually about a different guidance document (NMPA attachment pattern). R

Post-process the converted markdown, preserving base64 content images.

Compress image and return as base64 data URI HTML <img> tag.

Compare fetched data with existing and return a diff report.

Extract base URL from the board list page URL.

Extract inline footnotes and collect them at end of document. Footnotes in FDA PDFs appear as lines starting with a number followed by text l

Extract domain from 'site:domain.com' in query, if present.

Extract body content from an _unmatched/ file (skip comment headers).

Fix list items where the bullet is on one line and content on the next.

Convert YYYYMMDD to YYYY-MM-DD.

Get slugs for fulltext files changed in this PR (git diff vs base).

Get current harmonised standards count from local _index.json.

Extract outline level from paragraph XML (w:outlineLvl).

Check if bytes represent an OLE2 .doc file (not .docx).

Check if bytes represent a .docx (ZIP-based) file.

Load existing standards.json into a dict keyed by standard number.

Normalize announcement number to group siblings from the same notice.

Convert roman numeral section headers to proper markdown headings.

Map Chinese status labels to canonical English flags. 现行 -> active 即将实施 -> upcoming (still active) 其它 -> other

Parse announcements page, filter for medical-device-related entries. HSA announcement structure: <a href="/announcements/..."><div><h3>Title<

Parse Atom XML feed into list of entry dicts.

Parse article blocks from EC Latest Updates page.

Parse news entries from CDRH News page.

Fallback regex parser.

Parse EUR-Lex pages with flattened body format (no eli-container). Some consolidated regulations (e.g. CLP) serve content directly under <bod

Extract guidance documents with GN/GL identifiers and dates. Looks for patterns like: - GN-15-R13 Guidance on Medical Device Product

Extract meaningful links from an HTML page.

Parse one listing page. Returns (rows, page_info). The NIFDC page structures data across **multiple** ``<table>`` elements, one per classific

Extract all MDCG document references from EC page HTML. The EC page uses <tr class="ecl-table__row"> with cells: - data-ecl-table-h

Parse MFDS English board page (supports m_40, m_41, m_61).

Parse PMDA Japanese recall table page. Table columns: [0] recall_num, [1] pub_date, [2] device_type, [3] generic_name, [4] product_na

Parse PMDA table-based pages (safety info, precautions, alerts).

Parse SFDA weekly alert page. PDF links have empty text -- WU number and date are encoded in the URL-encoded href path, e.g.:

Parse PMDA What's New page -- list items with date, category, and title.

Rejoin lines that were broken by PDF extraction. If a line ends with a lowercase letter, comma, or similar continuation character and the nex

Remove cover page, preface, and preamble -- everything before the first real section. Strategy: find the FDA boilerplate 'This guidance represent

Remove standalone page numbers (single number on its own line).

Remove Table of Contents block.

Resolve human-readable source name from URL domain.

Check if a local path was created for this URL (via cache).

Query CELLAR SPARQL for the latest consolidated version date of CID 2021/1182. Returns date string like '20260407' or empty string on failure

Check if table contains any meaningful text.

Dispatch web search to Vertex AI (preferred), fallback to Google CSE.

Apply document-specific fixes to clean up severe OCR and layout conversion issues seen in certain MDR and MDCG guidance PDFs (e.g. 2019-16).

Batch convert PDFs listed in an _index.json file. Only converts entries that: 1. Have a source_url pointing to a PDF 2. Don't alread

Group translatable segments into API-friendly batches.

Build info list from .zh.md files with frontmatter data.

Build YAML front matter metadata from WordPress doc.

Build one RegulatoryNewsItem from a SINGLE detection item.

Build {l1_code: {l2_code: [items]}} structure.

Call LLM to generate bilingual summary for a SINGLE news item.

Guess category from title keywords.

Check URL accessibility. Returns status string.