Strip markdown fences, find outermost {…}, parse JSON.

Convert markdown text to HTML. Falls through to escaped text on error.

Return a codex_bin command list that runs *script* via bash -c. The ``--`` serves as argv0 so the provider's arguments (``exec --json …``) la

Build work items with the given chunk counts.

Build a minimal RawManpage for test use.

Check if subset's description mentions any of the superset's extra flags. This guards against false-positive subset merges that occur when the LL

Build file-backed read-only CachingStore instances for tests.

Return the (distro, release) pairs snapshotted at app startup. The DB is read-only and baked into the Docker image in prod, so the snapshot i

return the index of the next item returned by next()

Set the total number of batches (called once after grouping).

Extract NAME + SYNOPSIS + first paragraph of DESCRIPTION as preamble. The result is capped at ``_MAX_PREAMBLE_CHARS`` so the chunk budget never

Return a bash snippet that emits JSONL events mimicking codex output.

Store where 'uonly' is ubuntu-only, 'aonly' is arch-only, and 'both' exists in both distros.

Update batch counters and stats from a single file result.

Inject *store* into Flask's app extensions so ``get_store()`` returns it instead of opening a real database. Also refreshes ``STARTUP_DISTROS

Return row counts for core tables.

Return distinct (distro, release) pairs from manpages.

Return whether *source* exists in parsed_manpages.

Resolve a manpage source path to an external URL, or None.

Add line numbers to every line of text. Returns (numbered_text, original_lines) where original_lines is a dict mapping 1-indexed line numbers

Parse a TSV file, yielding rows as tuples.

Parse and validate a raw LLM response string. Returns (data_dict, raw_content). Raises ExtractionError on parse failure or validation failure

Fix universally-invalid field combinations. - positional + flags: clear positional - prefix without positional: clear prefix - prefix out

Reject an LLM extraction whose line spans look pathological. Raises ``ExtractionError`` when the sum of all option spans exceeds ``_MAX_COVER

Validate that *source* has the ``distro/release/section/name.section.gz`` format. Raises ``errors.InvalidSourcePath`` on failure.

Return True if *name* matches a distro in the cached distros list.

Parse a mode value into (mode, model). Returns ("llm", "<model>"). Raises ValueError on invalid input.

Run --diff A..B mode: compare two extractors on each file.

Remove options with duplicate (short+long) flag sets (from chunk overlap). When duplicates exist, keep the entry with the longest description so