MCPcopy Index your code
hub / github.com/Python3WebSpider/ProxyPool / get_proxy

Function get_proxy

proxypool/processors/server.py:89–130  ·  view source on GitHub ↗

get a random proxy, can query the specific sub-pool according the (redis) key if PROXY_RAND_KEY_DEGRADED is set to True, will get a universal random proxy if no proxy found in the sub-pool can pass a `count` parameter to get multiple random proxies at once can pass an `area` paramet

()

Source from the content-addressed store, hash-verified

87@app.route('/random')
88@auth_required
89def get_proxy():
90 """
91 get a random proxy, can query the specific sub-pool according the (redis) key
92 if PROXY_RAND_KEY_DEGRADED is set to True, will get a universal random proxy if no proxy found in the sub-pool
93 can pass a `count` parameter to get multiple random proxies at once
94 can pass an `area` parameter to only get proxies from a country (iso code, e.g. CN)
95 :return: get a random proxy
96 """
97 key = get_request_key()
98 count = request.args.get('count', type=int)
99 area = request.args.get('area')
100 conn = get_conn()
101 # return conn.random(key).string() if key else conn.random().string()
102 if area:
103 # area filtering needs the candidate set first, then filter by country
104 candidates = conn.all(key) if key else conn.all()
105 candidates = filter_proxies_by_area(candidates, area)
106 if not candidates and key and PROXY_RAND_KEY_DEGRADED:
107 candidates = filter_proxies_by_area(conn.all(), area)
108 if not candidates:
109 raise PoolEmptyException
110 if count and count > 1:
111 count = min(count, len(candidates))
112 return '\n'.join(proxy.string() for proxy in sample(candidates, count))
113 return choice(candidates).string()
114 if count and count > 1:
115 # return multiple random proxies, one per line
116 try:
117 proxies = conn.randoms(count, key) if key else conn.randoms(count)
118 except PoolEmptyException:
119 if key and PROXY_RAND_KEY_DEGRADED:
120 proxies = conn.randoms(count)
121 else:
122 raise
123 return '\n'.join(proxy.string() for proxy in proxies)
124 if key:
125 try:
126 return conn.random(key).string()
127 except PoolEmptyException:
128 if not PROXY_RAND_KEY_DEGRADED:
129 raise
130 return conn.random().string()
131
132
133@app.route('/all')

Callers

nothing calls this directly

Calls 7

get_request_keyFunction · 0.85
get_connFunction · 0.85
filter_proxies_by_areaFunction · 0.85
allMethod · 0.80
stringMethod · 0.80
randomsMethod · 0.80
randomMethod · 0.80

Tested by

no test coverage detected