MCPcopy Create free account
hub / github.com/apify/crawlee-python / _get_parser

Function _get_parser

src/crawlee/_utils/sitemap.py:211–216  ·  view source on GitHub ↗

Create appropriate parser based on content type and URL.

(content_type: str = '', url: str | None = None)

Source from the content-addressed store, hash-verified

209
210
211def _get_parser(content_type: str = '', url: str | None = None) -> _XmlSitemapParser | _TxtSitemapParser:
212 """Create appropriate parser based on content type and URL."""
213 if 'text/plain' in content_type.lower() or (url and URL(url).path.endswith('.txt')):
214 return _TxtSitemapParser()
215 # Default to XML parser for most cases
216 return _XmlSitemapParser()
217
218
219def _get_origin_url(source: SitemapSource) -> str:

Callers 2

_process_raw_sourceFunction · 0.85

Calls 2

_TxtSitemapParserClass · 0.85
_XmlSitemapParserClass · 0.85

Tested by

no test coverage detected