Function GetRobotsForURL

robots/robots.go:82–95 · view source on GitHub ↗

(target *url.URL, factory client.ClientFactory)

Source from the content-addressed store, hash-verified

source not stored for this graph (policy: none)

AddRobotsFilterMethod · 0.92

SeedFromRobotsMethod · 0.92

TestGetRobotsForURLFunction · 0.85

ParseRobotsTxtFunction · 0.85

GetMethod · 0.65

RequestURLMethod · 0.65

TestGetRobotsForURLFunction · 0.68