Helper to unify [reference labels].
(string: str)
| 251 | |
| 252 | |
| 253 | def normalizeReference(string: str) -> str: |
| 254 | """Helper to unify [reference labels].""" |
| 255 | # Trim and collapse whitespace |
| 256 | # |
| 257 | string = re.sub(r"\s+", " ", string.strip()) |
| 258 | |
| 259 | # In node v10 'ẞ'.toLowerCase() === 'Ṿ', which is presumed to be a bug |
| 260 | # fixed in v12 (couldn't find any details). |
| 261 | # |
| 262 | # So treat this one as a special case |
| 263 | # (remove this when node v10 is no longer supported). |
| 264 | # |
| 265 | # if ('ẞ'.toLowerCase() === 'Ṿ') { |
| 266 | # str = str.replace(/ẞ/g, 'ß') |
| 267 | # } |
| 268 | |
| 269 | # .toLowerCase().toUpperCase() should get rid of all differences |
| 270 | # between letter variants. |
| 271 | # |
| 272 | # Simple .toLowerCase() doesn't normalize 125 code points correctly, |
| 273 | # and .toUpperCase doesn't normalize 6 of them (list of exceptions: |
| 274 | # İ, ϴ, ẞ, Ω, K, Å - those are already uppercased, but have differently |
| 275 | # uppercased versions). |
| 276 | # |
| 277 | # Here's an example showing how it happens. Lets take greek letter omega: |
| 278 | # uppercase U+0398 (Θ), U+03f4 (ϴ) and lowercase U+03b8 (θ), U+03d1 (ϑ) |
| 279 | # |
| 280 | # Unicode entries: |
| 281 | # 0398;GREEK CAPITAL LETTER THETA;Lu;0;L;;;;;N;;;;03B8 |
| 282 | # 03B8;GREEK SMALL LETTER THETA;Ll;0;L;;;;;N;;;0398;;0398 |
| 283 | # 03D1;GREEK THETA SYMBOL;Ll;0;L;<compat> 03B8;;;;N;GREEK SMALL LETTER SCRIPT THETA;;0398;;0398 |
| 284 | # 03F4;GREEK CAPITAL THETA SYMBOL;Lu;0;L;<compat> 0398;;;;N;;;;03B8 |
| 285 | # |
| 286 | # Case-insensitive comparison should treat all of them as equivalent. |
| 287 | # |
| 288 | # But .toLowerCase() doesn't change ϑ (it's already lowercase), |
| 289 | # and .toUpperCase() doesn't change ϴ (already uppercase). |
| 290 | # |
| 291 | # Applying first lower then upper case normalizes any character: |
| 292 | # '\u0398\u03f4\u03b8\u03d1'.toLowerCase().toUpperCase() === '\u0398\u0398\u0398\u0398' |
| 293 | # |
| 294 | # Note: this is equivalent to unicode case folding; unicode normalization |
| 295 | # is a different step that is not required here. |
| 296 | # |
| 297 | # Final result should be uppercased, because it's later stored in an object |
| 298 | # (this avoid a conflict with Object.prototype members, |
| 299 | # most notably, `__proto__`) |
| 300 | # |
| 301 | return string.lower().upper() |
| 302 | |
| 303 | |
| 304 | LINK_OPEN_RE = re.compile(r"^<a[>\s]", flags=re.IGNORECASE) |