MCPcopy Create free account

hub / github.com/cld378632668/SinaWeiboCrawler / functions

Functions84 in github.com/cld378632668/SinaWeiboCrawler

↓ 37 callersMethodget
根据URL,get网页 @param url @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:219
↓ 9 callersMethode
(int j, int m)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:91
↓ 8 callersMethodgetHTML
(String url, JTextArea jta)
src/cn/edu/xmu/cld/WeiboCrawler2/HTML.java:33
↓ 6 callersMethodparse
(String html)
src/cn/edu/xmu/cld/WeiboCrawler2/HTMLParser.java:40
↓ 5 callersMethodencodeAccount
(String account)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:565
↓ 5 callersMethodgetFilePath
()
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:55
↓ 5 callersMethodopen
()
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:46
↓ 4 callersMethodgetHTMLbyProxy
(String targetURL, String hostName, int port, JTextArea jta)
src/cn/edu/xmu/cld/WeiboCrawler2/HTML.java:64
↓ 3 callersMethodd
(int j, int k)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:97
↓ 3 callersMethodhex_sha1
(String j)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:19
↓ 3 callersMethodmakeNonce
(int len)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:578
↓ 3 callersMethodwrite2txt
(Vector<String> vector, String savePath)
src/cn/edu/xmu/cld/WeiboCrawler2/FileOperation.java:28
↓ 2 callersMethodencode
(String psw, String servertime, String nonce)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:13
↓ 2 callersMethodgetCookieStore
(CookieStore cookiestore)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:32
↓ 2 callersMethodgetJs
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:119
↓ 2 callersMethodhtml2String
由html文件得到微博 @param html @return @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/FileOperation.java:65
↓ 2 callersMethodlogin
(String username, String password )
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:58
↓ 2 callersMethodpreLogin
新浪微博预登录,获取密码加密公钥 @param unameBase64 @return 返回从结果获取的参数的哈希表 @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:236
↓ 1 callersMethoda
(int k, int j, int m, int l)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:70
↓ 1 callersMethodaction
()
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:316
↓ 1 callersMethodb
(int[] A, int r)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:33
↓ 1 callersMethodc
(int j)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:86
↓ 1 callersMethodcrawl
此函数调用底层函数,执行具体的爬取微博的过程 并保存微博搜索结果页面html文件 @param totalPage @param searchword @throws ClientProtocolException @throws URISyntaxException @throws IOExcep
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:142
↓ 1 callersMethodcreate
(HttpContext context)
src/cn/edu/xmu/cld/WeiboCrawler2/HTML.java:103
↓ 1 callersMethodcreate
(HttpContext context)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:216
↓ 1 callersMethoddump
打印页面 @param entity @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:270
↓ 1 callersMethodexcute
excute()是类的主函数,主要逻辑: allIPs = getAllProxyIPs(ipLibURL) 从代理ip网站首页网址获取所有代理IP validIPs = getValidProxyIPs(allIPs) 从所有代理IP中获取有效代理Ip iPsSupportedWeibo = ne
src/cn/edu/xmu/cld/WeiboCrawler2/ProxyIP.java:337
↓ 1 callersMethodexcute
(String[] args, JTextArea jta)
src/cn/edu/xmu/cld/WeiboCrawler2/Crawler.java:43
↓ 1 callersMethodf
(String m, int r)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:101
↓ 1 callersMethodfilterIPsSupportSpeWebsite
过滤出能用于连接特定网址的proxyIps @param IPs - the Vector<String> contains all valid IPs @param outfilePath @return @throws ClientProtocolException
src/cn/edu/xmu/cld/WeiboCrawler2/ProxyIP.java:269
↓ 1 callersMethodgetAllProxyIPs
Get all unverified proxy IP in all IP library links. @param ipLibURL specified URL "http://www.youdaili.cn/" @return a String Vector contains all unv
src/cn/edu/xmu/cld/WeiboCrawler2/ProxyIP.java:101
↓ 1 callersMethodgetCommentNum
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:411
↓ 1 callersMethodgetDate
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:435
↓ 1 callersMethodgetForwardNum
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:387
↓ 1 callersMethodgetHTML
由url得到html @param url @return html @throws URISyntaxException @throws ClientProtocolException @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:203
↓ 1 callersMethodgetIPsPageLinks
Find all IP library links on the homepage of "http://www.xici.net.co/". @param ipLibURL specified URL "http://www.youdaili.cn/" @return a
src/cn/edu/xmu/cld/WeiboCrawler2/ProxyIP.java:66
↓ 1 callersMethodgetLines
(String path)
src/cn/edu/xmu/cld/WeiboCrawler2/FileOperation.java:13
↓ 1 callersMethodgetMyTime
()
src/cn/edu/xmu/cld/WeiboCrawler2/Crawler.java:34
↓ 1 callersMethodgetMyTime
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:172
↓ 1 callersMethodgetParaFromResult
从新浪返回的结果字符串中获得参数 @param result @return
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:248
↓ 1 callersMethodgetPraisedNum
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:366
↓ 1 callersMethodgetProxyIPs
获取一个HTML页面中所有的IPs There are proxy ips which haven't been verified on the page like ?? http://www.youdaili.cn/Daili/guonei/1843.html, use regex to ma
src/cn/edu/xmu/cld/WeiboCrawler2/ProxyIP.java:30
↓ 1 callersMethodgetServerTime
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:588
↓ 1 callersMethodgetUserName
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:448
↓ 1 callersMethodgetUserid
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:313
↓ 1 callersMethodgetValidProxyIPs
Test all proxy IPs and select the valid ones. @param a String Vector which contains all proxy IPs @return a String Vector which contains all valid IP
src/cn/edu/xmu/cld/WeiboCrawler2/ProxyIP.java:176
↓ 1 callersMethodgetWeiboInfo
(String htmlString)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:479
↓ 1 callersMethodgetWeiboid
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:299
↓ 1 callersMethodh
(int[] l)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:23
↓ 1 callersMethodhtmltoString
由html文件得到微博 @param html @return @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:284
↓ 1 callersMethodisExistResult
是否存在指定页数的搜索结果页面 @param html @return
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:593
↓ 1 callersMethodisVerification
(String html)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:576
↓ 1 callersMethodrunBroswer
用默认浏览器打开指定网址 @param url @throws URISyntaxException @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:188
↓ 1 callersMethodsetDisplayName
(String displayName)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:17
↓ 1 callersMethodsetUserName
(String userName)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:23
↓ 1 callersMethodsetUserPass
(String userPass)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:29
↓ 1 callersMethodsplitHTML
(String html)
src/cn/edu/xmu/cld/WeiboCrawler2/HTMLParser.java:20
↓ 1 callersMethodwrite2txt
(String searchword, String dirPath, String saveTXTPath)
src/cn/edu/xmu/cld/WeiboCrawler2/HTMLParser.java:160
↓ 1 callersMethodwriteString
把String写到本地文件 @param s @param savePath @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/FileOperation.java:48
↓ 1 callersMethodwriteString
把String写到本地文件 @param s @param savePath @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:270
↓ 1 callersMethodwriteVector
把某关键字搜索到的微博全写到文件中去 这里简单写成了txt格式 @param vector @param savePath @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:521
↓ 1 callersMethodwriteVector2xml
(Vector<String> vector, String saveXMLPath)
src/cn/edu/xmu/cld/WeiboCrawler2/HTMLParser.java:201
MethodAddW
(String u, String p, String text, byte[] pic)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:380
MethodJWindowsFrame
()
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:131
MethodOpenFile
()
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:38
MethodShare
(String u, String p, String content, String pic, String surl)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:281
MethodSinaSSOEncoder
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/SinaSSOEncoder.java:9
MethodactionPerformed
(ActionEvent e)
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:320
MethodgetDisplayName
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:14
MethodgetRealHTML
(String htmlPath)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/ReParser.java:27
MethodgetUserName
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:20
MethodgetUserPass
()
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiBoUser.java:26
MethodgetWeiboSentence
(String iniWeibo)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:327
MethodisExistResult
是否存在指定页数的搜索结果页面 @param html @return
src/cn/edu/xmu/cld/WeiboCrawler2/Crawler.java:23
Methodmain
(String[] args)
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:563
Methodmain
(String[] args)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:49
Methodmain
@param args @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:50
MethodreadFileImage
(String filename)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/Sina.java:551
Methodrun
()
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:473
MethodshowData
(String s)
src/cn/edu/xmu/cld/WeiboCrawler2/JWindowsFrame.java:311
Methodutf8Parser
(String utf8)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/ReParser.java:19
Methodvalidate
(Cookie cookie, CookieOrigin origin)
src/cn/edu/xmu/cld/WeiboCrawler2/HTML.java:105
Methodvalidate
(Cookie cookie, CookieOrigin origin)
src/cn/edu/xmu/cld/WeiboCrawler2/useVersion2014/WeiboCrawler3.java:218
MethodwriteVector
把某关键字搜索到的微博全写到文件中去 这里简单写成了txt格式 @param vector @param savePath @throws IOException
src/cn/edu/xmu/cld/WeiboCrawler2/FileOperation.java:86