MCPcopy
hub / github.com/unclecode/crawl4ai / crawl

Method crawl

crawl4ai/crawler_strategy.py:66–77  ·  view source on GitHub ↗
(self, url: str)

Source from the content-addressed store, hash-verified

64 self.use_cached_html = use_cached_html
65
66 def crawl(self, url: str) -> str:
67 data = {
68 "urls": [url],
69 "include_raw_html": True,
70 "forced": True,
71 "extract_blocks": False,
72 }
73
74 response = requests.post("http://crawl4ai.uccode.io/crawl", json=data)
75 response = response.json()
76 html = response["results"][0]["html"]
77 return sanitize_input_encode(html)
78
79class LocalSeleniumCrawlerStrategy(CrawlerStrategy):
80 def __init__(self, use_cached_html=False, js_code=None, **kwargs):

Callers

nothing calls this directly

Calls 1

sanitize_input_encodeFunction · 0.85

Tested by

no test coverage detected