MCPcopy Create free account

hub / github.com/aidenwang9867/Weibo-User-Depression-Detection-Dataset / functions

Functions148 in github.com/aidenwang9867/Weibo-User-Depression-Detection-Dataset

⨍ Functions 148◇ Types & classes 16
↓ 8 callersMethodget_filepath
获取结果文件路径
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:829
↓ 8 callersMethodget_filepath
获取结果文件路径
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:572
↓ 8 callersMethodhandle_garbled
处理乱码
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:127
↓ 8 callersMethodhandle_garbled
处理乱码
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:111
↓ 7 callersMethodstart
运行爬虫
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1228
↓ 6 callersFunctionredis_init
(spider_name, urls)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/redis_init.py:14
↓ 5 callersMethodhandle_html
处理html
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:117
↓ 5 callersMethodinsert_item
(collection, item)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/pipelines.py:31
↓ 4 callersMethodhandle_html
处理html
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:101
↓ 4 callersMethodis_date
判断日期格式是否正确
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:98
↓ 3 callersMethoddownload_one_file
下载单个文件(图片/视频)
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:680
↓ 3 callersMethoddownload_one_file
下载单个文件(图片/视频)
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:430
↓ 3 callersMethodstr_to_time
将字符串转换成时间类型
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:109
↓ 3 callersFunctiontime_fix
(time_string)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/spiders/utils.py:7
↓ 2 callersMethoddownload_files
下载文件(图片/视频)
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:720
↓ 2 callersMethoddownload_files
下载文件(图片/视频)
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:470
↓ 2 callersMethodextract_picture_urls
提取微博原始图片url
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:584
↓ 2 callersMethodextract_picture_urls
提取微博原始图片url
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:334
↓ 2 callersFunctionextract_weibo_content
(weibo_html)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/spiders/utils.py:38
↓ 2 callersMethodget_json
获取网页中json数据
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:178
↓ 2 callersMethodget_long_weibo
获取长原创微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:395
↓ 2 callersMethodget_long_weibo
获取长原创微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:134
↓ 2 callersMethodinfo_to_mongodb
将爬取的信息写入MongoDB数据库
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:975
↓ 2 callersFunctioninsert_cookie
insert cookie to database :param username: username of weibo account :param password: password of weibo account :param cookie_str: co
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/account/db_utils.py:16
↓ 2 callersMethodmysql_create
创建MySQL数据库或表
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1007
↓ 2 callersMethodmysql_create
创建MySQL数据库或表
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:757
↓ 2 callersMethodmysql_create_table
创建MySQL表
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1029
↓ 2 callersMethodmysql_insert
向MySQL表插入或更新数据
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1039
↓ 2 callersFunctionweibo_filter
(content, repstr='')
WU3D_crawler_preprocessing/Data_processing/Wash_data.py:7
↓ 2 callersFunctionweibo_filter
(content, repstr='')
WU3D_crawler_preprocessing/Data_processing/Tag.py:7
↓ 2 callersMethodwrite_data
将爬取到的信息写入文件或数据库
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1135
↓ 2 callersMethodwrite_data
将爬取到的信息写入文件或数据库
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:891
↓ 1 callersMethodcsv_helper
将指定信息写入csv文件
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:312
↓ 1 callersFunctionextract_comment_content
(comment_html)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/spiders/utils.py:63
↓ 1 callersMethodextract_user_info
提取用户信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:137
↓ 1 callersMethodfetch_proxy
(self)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/middlewares.py:52
↓ 1 callersMethodget_keyword_list
获取文件中的keyword
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:947
↓ 1 callersMethodget_long_retweet
获取长转发微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:430
↓ 1 callersMethodget_long_retweet
获取长转发微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:170
↓ 1 callersMethodget_nickname
获取用户昵称
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:235
↓ 1 callersMethodget_one_page
获取第page页的全部微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:791
↓ 1 callersMethodget_one_page
获取第page页的全部微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:543
↓ 1 callersMethodget_one_weibo
获取一条微博的全部信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:740
↓ 1 callersMethodget_one_weibo
获取一条微博的全部信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:490
↓ 1 callersMethodget_original_weibo
获取原创微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:414
↓ 1 callersMethodget_original_weibo
获取原创微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:153
↓ 1 callersMethodget_page_num
获取微博总页数
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:380
↓ 1 callersMethodget_page_num
获取微博总页数
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:121
↓ 1 callersMethodget_picture_urls
获取微博原始图片url
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:622
↓ 1 callersMethodget_picture_urls
获取微博原始图片url
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:372
↓ 1 callersMethodget_publish_place
获取微博发布位置
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:490
↓ 1 callersMethodget_publish_place
获取微博发布位置
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:240
↓ 1 callersMethodget_publish_time
获取微博发布时间
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:515
↓ 1 callersMethodget_publish_time
获取微博发布时间
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:265
↓ 1 callersMethodget_publish_tool
获取微博发布工具
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:547
↓ 1 callersMethodget_publish_tool
获取微博发布工具
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:297
↓ 1 callersMethodget_retweet
获取转发微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:440
↓ 1 callersMethodget_retweet
获取转发微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:180
↓ 1 callersFunctionget_uid
(name,new_header)
WU3D_crawler_preprocessing/Data_processing/UID_to_name.py:9
↓ 1 callersMethodget_user
获取用户信息、微博数、关注数、粉丝数
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:359
↓ 1 callersMethodget_user_config_list
获取文件中的微博id信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1194
↓ 1 callersMethodget_user_id
获取用户id，使用者输入的user_id不一定是正确的，可能是个性域名等，需要获取真正的user_id
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:346
↓ 1 callersMethodget_user_id
获取用户id
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:230
↓ 1 callersMethodget_user_info
获取用户信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:194
↓ 1 callersMethodget_video_url
获取微博视频url
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:649
↓ 1 callersMethodget_video_url
获取微博视频url
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:399
↓ 1 callersMethodget_weibo_content
获取微博内容
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:477
↓ 1 callersMethodget_weibo_content
获取微博内容
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:217
↓ 1 callersMethodget_weibo_footer
获取微博点赞数、转发数、评论数
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:561
↓ 1 callersMethodget_weibo_footer
获取微博点赞数、转发数、评论数
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:311
↓ 1 callersMethodget_weibo_info
获取微博信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1153
↓ 1 callersMethodget_weibo_info
获取微博信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:910
↓ 1 callersMethodhandle_download
处理下载相关操作
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:698
↓ 1 callersMethodhandle_download
处理下载相关操作
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:448
↓ 1 callersMethodinfo_to_mongodb
将爬取的信息写入MongoDB数据库
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:725
↓ 1 callersMethodinitialize_info
初始化爬虫信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1220
↓ 1 callersMethodinitialize_info
初始化爬虫信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:962
↓ 1 callersMethodis_date
判断日期格式是否正确
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:82
↓ 1 callersMethodis_original
判断微博是否为原创微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:469
↓ 1 callersMethodis_original
判断微博是否为原创微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:209
↓ 1 callersMethodis_pinned_weibo
判断微博是否为置顶微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:783
↓ 1 callersFunctionjudge
()
WU3D_crawler_preprocessing/Data_processing/Data_conversion.py:4
↓ 1 callersFunctionjudge
()
WU3D_crawler_preprocessing/Data_processing/Tag.py:20
↓ 1 callersMethodlogin
open login page and login :return: None
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/account/login.py:36
↓ 1 callersFunctionmain
()
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1244
↓ 1 callersFunctionmain
()
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:988
↓ 1 callersFunctionmain
()
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/account/login.py:67
↓ 1 callersFunctionmain
()
WU3D_crawler_preprocessing/Data_processing/Get_pic.py:51
↓ 1 callersMethodmysql_create_database
创建MySQL数据库
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:1015
↓ 1 callersMethodmysql_create_database
创建MySQL数据库
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:765
↓ 1 callersMethodmysql_create_table
创建MySQL表
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:779
↓ 1 callersMethodmysql_insert
向MySQL表插入或更新数据
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:789
↓ 1 callersMethodprint_one_weibo
打印一条微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:772
↓ 1 callersMethodprint_one_weibo
打印一条微博
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:524
↓ 1 callersMethodprint_user_info
打印微博用户信息
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:337
↓ 1 callersMethodrun
(self)
WU3D_crawler_preprocessing/Weibo_Spider/Spider2.0/weibospider/account/login.py:50
↓ 1 callersMethodstandardize_info
标准化信息，去除乱码
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:184
↓ 1 callersMethodstart
运行爬虫
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:969
↓ 1 callersMethodupdate_json_data
更新要写入json结果文件中的数据，已经存在于json中的信息更新为最新值，不存在的信息添加到data中
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/user_spider.py:936
↓ 1 callersMethodupdate_json_data
更新要写入json结果文件中的数据，已经存在于json中的信息更新为最新值，不存在的信息添加到data中
WU3D_crawler_preprocessing/Weibo_Spider/Spider1.0/topic_spider.py:686
next →1–100 of 148, ranked by callers