MCPcopy Create free account

hub / github.com/Germey/ProxyPool / functions

Functions47 in github.com/Germey/ProxyPool

⨍ Functions 47◇ Types & classes 9↳ Endpoints 3
↓ 9 callersMethodget
get proxies from redis
proxypool/db.py:13
↓ 8 callersFunctionget_page
(url, options={})
proxypool/utils.py:8
↓ 3 callersMethodrun
(self)
proxypool/schedule.py:133
↓ 2 callersFunctionget_conn
Opens a new redis connection if there is none yet for the current application context.
proxypool/api.py:10
↓ 2 callersFunctionget_proxy
()
examples/example_round_proxy.py:7
↓ 2 callersMethodis_over_threshold
judge if count is overflow.
proxypool/schedule.py:72
↓ 2 callersMethodpop
get proxy from right.
proxypool/db.py:27
↓ 2 callersMethodset_raw_proxies
(self, proxies)
proxypool/schedule.py:23
↓ 2 callersMethodtest
aio test all proxies.
proxypool/schedule.py:48
↓ 1 callersMethod__init__
(self)
proxypool/error.py:12
↓ 1 callersMethodadd_to_queue
(self)
proxypool/schedule.py:81
↓ 1 callersFunctioncrawl
(url, proxy)
examples/example.py:16
↓ 1 callersFunctioncrawl
(url, proxy)
examples/example_round_proxy.py:17
↓ 1 callersMethoddownload
(self)
proxypool/utils.py:44
↓ 1 callersMethoddownload_single_page
(self, url)
proxypool/utils.py:39
↓ 1 callersFunctionget_count_proxys
()
examples/example_round_proxy.py:12
↓ 1 callersFunctionget_proxy
()
examples/example.py:10
↓ 1 callersMethodget_raw_proxies
(self, callback)
proxypool/getter.py:25
↓ 1 callersFunctionmain
()
run.py:4
↓ 1 callersFunctionmain
()
examples/example.py:22
↓ 1 callersFunctionmain
()
examples/example_round_proxy.py:22
↓ 1 callersMethodput
add proxy to right top
proxypool/db.py:21
↓ 1 callersMethodtest_single_proxy
text one proxy, if valid, put them to usable_proxies.
proxypool/schedule.py:27
Method__init__
(self, host=HOST, port=PORT)
proxypool/db.py:7
Method__init__
(self)
proxypool/schedule.py:19
Method__init__
(self, threshold)
proxypool/schedule.py:66
Method__init__
(self, urls)
proxypool/utils.py:35
Method__init__
(self)
proxypool/error.py:3
Method__new__
(cls, name, bases, attrs)
proxypool/getter.py:13
Method__str__
(self)
proxypool/error.py:6
Method__str__
(self)
proxypool/error.py:15
Methodcheck_pool
If the number of proxies less than lower_threshold, add proxy
proxypool/schedule.py:120
Methodcrawl_daili66
(self, page_count=4)
proxypool/getter.py:69
Methodcrawl_data5u
(self)
proxypool/getter.py:83
Methodcrawl_ip181
(self)
proxypool/getter.py:33
Methodcrawl_kuaidaili
(self)
proxypool/getter.py:43
Methodcrawl_kxdaili
(self)
proxypool/getter.py:96
Methodcrawl_premproxy
(self)
proxypool/getter.py:107
Methodcrawl_xicidaili
(self)
proxypool/getter.py:56
Methodcrawl_xroxy
(self)
proxypool/getter.py:118
Methodflush
flush db
proxypool/db.py:43
Functionget_counts
Get the count of proxies
proxypool/api.py:35
Functionget_proxy
Get a proxy
proxypool/api.py:26
Methodhtmls
(self)
proxypool/utils.py:50
Functionindex
()
proxypool/api.py:21
Methodqueue_len
get length from queue.
proxypool/db.py:37
Methodvalid_proxy
Get half of proxies which in redis
proxypool/schedule.py:101