Login baidutieba by mobile

Login xiaohongshu by mobile

login zhihu website and keep webdriver login state

login bilibili website and keep webdriver login state

login weibo website and keep webdriver login state

login baidutieba website and keep webdriver login state

login kuaishou website and keep webdriver login state

login xiaohongshu website and keep webdriver login state

Main function

Main function

旧 GraphQL API 格式：camelCase 字段名。

V2 API 格式：snake_case 字段名，comment_id 为 int。

make save file name by store type Args: aid: aid extension_file_name: video filename with extension

make save file name by store type Args: picid: image id extension_file_name: video filename with extension

make save file name by store type Args: notice_id: notice id extension_file_name: image filename with extens

make save file name by store type Args: notice_id: notice id extension_file_name: video filename with extens

make save file name by store type Args: aweme_id: aweme id extension_file_name: image filename with extensio

make save file name by store type Args: aweme_id: aweme id extension_file_name: video filename with extensio

Move the slider to the right to complete the verification :param back_selector: Selector for the slider verification background image

Construct KuaiDaili HTTP instance Supports two environment variable naming formats: 1. Uppercase format: KDL_SECERT_ID, KDL_SIGNATURE, KD

Construct WanDou HTTP instance Supports two environment variable naming formats: 1. Uppercase format: WANDOU_APP_KEY 2. Lowercase for

Parse KuaiDaili IP information Args: proxy_info: Returns:

Parse note information from Xiaohongshu note URL Args: url: "https://www.xiaohongshu.com/explore/66fad51c000000001b0224b8?xsec_token=

Check if login status is still valid Returns:

get a note to check if login state is ok

get a note to check if login state is ok

Check if login state is still valid Uses Cookie detection instead of API calls to avoid detection Args: browser_c

get a note to check if login state is ok

Check if login state is still valid by querying self user info Returns: bool: True if logged in, False otherwise

If the login dialog box does not pop up automatically, we will manually click the login button

Process single Python file Args: file_path: File path project_root: Project root directory dry_run: Check only witho

Query self user info to check login state Returns: Dict: User info if logged in, None otherwise

Wrapper for httpx common request method with response handling Args: method: Request method url: Request URL

解析抖音短链接,获取重定向后的真实URL Args: short_url: 短链接,如 https://v.douyin.com/iF12345ABC/ Returns: 重定向后的完整URL

save image to local Args: picid: image id pic_content: image content extension_file_name: image

save image to local Args: notice_id: notice id pic_content: image content extension_file_name: i

save image to local Args: aweme_id: aweme id pic_content: image content extension_file_name: ima

save video to local Args: aid: aid video_content: video content extension_file_name: video filen

save video to local Args: notice_id: notice id video_content: video content extension_file_name:

save video to local Args: aweme_id: aweme id video_content: video content extension_file_name: v

Search for notes and retrieve their comment information.

search bilibili video with keywords in normal mode :return:

KuaiShou web search api :param keyword: search keyword :param pcursor: limite page curson :param search_session_id: s

DouYin Web Search API :param keyword: :param offset: :param search_channel: :param sort_type: :param

Set the value of a key in the cache :param key: :param value: :param expire_time: :return:

zhihu sign algorithm Args: url: request url with query string cookies: request cookies with d_c0 key Returns:

Add current timestamp to request parameters, sort keys in dictionary order, then URL encode the parameters and combine with salt to g

使用 xhshow 纯算法生成完整签名请求头 Args: uri: API path data: Request data (GET params dict or POST payload dict) cookie_str: Coo

Start crawler process

Start broadcast task

Zhihu content DB storage implementation Args: comment_item: comment item dict

Bilibili comment DB storage implementation Args: comment_item: comment item dict

Weibo content DB storage implementation Args: comment_item: comment item dict Returns:

tieba content DB storage implementation Args: comment_item: comment item dict

Kuaishou comment DB storage implementation Args: comment_item: comment item dict

Store comment to MongoDB Args: comment_item: Comment data

Douyin comment DB storage implementation Args: comment_item: comment item dict

Store comment to MongoDB Args: comment_item: Comment data

Zhihu content DB storage implementation Args: content_item: content item dict

Bilibili content DB storage implementation Args: content_item: content item dict

Weibo content DB storage implementation Args: content_item: content item dict Returns:

tieba content DB storage implementation Args: content_item: content item dict

Kuaishou content DB storage implementation Args: content_item: content item dict

Store note content to MongoDB Args: content_item: Note content data

Douyin content DB storage implementation Args: content_item: content item dict

Store video content to MongoDB Args: content_item: Video content data

Creator profile is no longer persisted (teaching version: anti-harassment).

Store creator information to MongoDB Args: creator_item: Creator data

Bilibili dynamic DB storage implementation Args: dynamic_item: dynamic item dict

store content Args: image_content_item: Returns:

store content Args: image_content_item: Returns:

store content Args: image_content_item: Returns:

store content Args: video_content_item: Returns:

store content Args: video_content_item: Returns:

store content Args: video_content_item: Returns: