to string :return: <host>:<port>

get all proxies :return: list of proxies

convert list of str to valid proxies or proxy :param data: :return:

add proxy and set it to init score :param proxy: proxy, ip:port, like 8.8.8.8:88 :param score: int score :return: res

get count of proxies :return: count, int

decrease score of proxy, if small than PROXY_SCORE_MIN, delete it :param proxy: proxy :return: new score

extract host and port from a proxy with authentication

filter proxies by country iso code (e.g. 'CN', 'US'), case-insensitive; proxies whose country cannot be resolved are excluded :param prox

get redis client object :return:

look up the ISO country code (e.g. 'CN', 'US') for an ip address :param ip: ip address string :return: uppercase iso code, or None if unk

read the `key` query parameter and validate its format; reject unexpected characters to avoid redis key probing/injection :return: valida

check this string is within ip format

check this string is within proxy format

get random proxy firstly try to get proxy with max score if not exists, try to get proxy by rank if not exists, raise

get a batch of random proxies firstly try to get proxies with max score, if not enough, get proxies by rank (score from high

test main method :return:

if proxy exists :param proxy: proxy :return: if exists, bool

set proxy to max score :param proxy: proxy :return: new score

used for parse html

run crawlers to get proxy :return:

test single proxy :param proxy: Proxy object :param session: shared aiohttp session :return:

to string, for print :return:

get batch of proxies :param cursor: scan cursor :param count: scan count :return: list of proxies

use proxy to crawl page :param url: page url :param proxy: proxy, such as 8.8.8.8:8888 :return: html

crawl main method

override crawl main method add headers

get random proxy from proxypool :return: proxy

if proxypool if full return: bool

main method, entry point :return: none

parse html file to get proxies :return:

parse html file to get proxies :return:

test all proxies in batches, reusing a single aiohttp session :return:

init redis client :param host: redis host :param port: redis port :param password: redis password :param conn

init redis

init db and crawlers

init urls

proxypool is used out :return:

get the count of proxies :return: count, int

get a random proxy, can query the specific sub-pool according the (redis) key if PROXY_RAND_KEY_DEGRADED is set to True, will get a universal

get all proxies, optionally filtered by `area` (country iso code, e.g. CN) :return: all proxies

get home page, you can define your own templates :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

parse html file to get proxies :return:

run getter

run server for api

run tester