MCPcopy Create free account

hub / github.com/Lichanglu/pholcus / functions

Functions266 in github.com/Lichanglu/pholcus

MethodLogInfo
LogError logs normal info.
common/mlog/filelog.go:91
FunctionLogInst
LogInst get the singleton filelog object.
common/mlog/filelog.go:25
FunctionMakeHash
string to hash
common/util/util.go:106
FunctionMakeMd5
制作特征值方法二
common/util/util.go:126
FunctionMakeUnique
制作特征值方法一
common/util/util.go:120
MethodMinSizeHint
()
pholcus/gui/logview.go:44
FunctionNew
()
pholcus/crawler/crawl.go:24
FunctionNew
()
common/deduplicate/deduplicate.go:16
FunctionNew
()
pipeline/pipeline.go:30
FunctionNewCollector
()
pipeline/collector/collector.go:24
FunctionNewConfig
()
common/config/config.go:18
FunctionNewDataCell
(ruleName string, data map[string]string, url string, parentUrl string, downloadTime string)
pipeline/collector/datacell.go:6
FunctionNewHttpDownloader
()
downloader/downloader_http.go:32
FunctionNewQueue
(size int)
common/queue/queue.go:8
FunctionNewRequest
NewRequest returns initialized Request object. The respType is json, jsonp, html, text
downloader/context/request.go:45
FunctionNewResponse
NewResponse returns initialized Response object.
downloader/context/response.go:38
MethodOpen
Open makes log open.
common/mlog/mlog.go:24
FunctionPhoSocketAccept
* *@服务端用 * *PhoSocketAccept()为幽灵蛛socket同意连接函数 *ln为一个监听句柄 *输出类型为net.Conn,一个握手连接,下一步可以进行接收,发送 */
pholcus/socket/PhoSocketTest.go:58
FunctionPhoSoketDial
* *@客户端用 * *PhoSoketDial()为幽灵蛛socket拨号函数,请求服务端 *PhoSocketServer为预定义常量:server:port *输出类型为net.Conn,一个握手连接,下一步可以进行接收,发送 */
pholcus/socket/PhoSocket.go:43
FunctionPhoSoketLisent
建立连接 func 接受 func 发送 * *@服务端用 * *PhoSoketLisent()为幽灵蛛socket监听函数 *PhoSocketServer为预定义常量:server:port *输出类型为net.Listener,一个监听句柄 */
pholcus/socket/PhoSocketTest.go:28
MethodPrintf
(format string, v ...interface{})
reporter/report.go:16
MethodPrintln
(v ...interface{})
reporter/report.go:21
MethodPrintln
Println output the str to os.Stderr.
common/mlog/strace.go:37
MethodPush
(req *context.Request)
scheduler/src_manage.go:33
MethodPush
(req *context.Request)
scheduler/scheduler.go:38
MethodPush
(req *context.Request)
pholcus/pholcus.go:19
MethodReceive
(req context.Request)
pholcus/pholcus.go:49
MethodResetHtmlParser
GetHtmlParser returns goquery object binded to target crawl result.
downloader/context/response.go:125
MethodRowCount
Called by the TableView from SetModel and every time the model publishes a RowsReset event.
pholcus/gui/guispider.go:56
FunctionRun
()
pholcus/gui/guimain.go:19
MethodSectionContent
(section string)
common/config/config.go:247
MethodSectionContents
()
common/config/config.go:251
MethodSectionGetDuration
(section string, key string)
common/config/config.go:232
MethodSectionGetSlice
(section string, key string, separator string)
common/config/config.go:236
MethodSectionHas
(section string, key string)
common/config/config.go:193
MethodSectionSet
(section string, key string, value string)
common/config/config.go:210
MethodSections
()
common/config/config.go:182
MethodSetChecked
Called by the TableView when the user toggled the check box of a given row.
pholcus/gui/guispider.go:86
MethodSetOutsource
(serve bool)
pholcus/pholcus.go:37
MethodSetRequest
SetRequest saves request oject of self page.
downloader/context/response.go:92
MethodSetTemp
(key string, value interface{})
downloader/context/request.go:248
MethodSetUrl
(url string)
downloader/context/request.go:191
MethodShowOutFeild
获取任意规则采集语义字段
spiders/spider/spider.go:73
MethodSizeHint
()
pholcus/gui/logview.go:48
MethodSort
Called by the TableView to sort the model.
pholcus/gui/guispider.go:104
MethodStart
func NewSpider() *Spider { sp := new(Spider) sp.RuleTree = &RuleTree{ Nodes: make(map[string]*Rule), } return sp }
spiders/spider/spider.go:30
MethodStart
任务执行入口
pholcus/crawler/crawl.go:41
MethodStart
()
pipeline/pipeline.go:12
MethodStart
()
pipeline/pipeline.go:50
FunctionStartConf
StartConf is used in Spider for initialization at first time.
common/etc_config/etc_config.go:31
FunctionStraceInst
StraceInst get the singleton strace object.
common/mlog/strace.go:21
MethodStringWithMeta
()
common/config/config.go:102
MethodSwap
(i, j int)
pholcus/gui/guispider.go:144
MethodTryOutsource
()
downloader/context/request.go:231
MethodUse
(spiderId int)
scheduler/src_manage.go:39
MethodValue
Called by the TableView when it needs the text to display for a given cell.
pholcus/gui/guispider.go:61
MethodWndProc
(hwnd win.HWND, msg uint32, wParam, lParam uintptr)
pholcus/gui/logview.go:84
FunctionXML2mapstr
simple xml to string support utf8
common/util/util.go:80
Functioninit
()
reporter/report.go:28
Functioninit
()
pholcus/pholcus.go:56
Functioninit
()
config/config.go:55
Functionmain
()
main.go:7
Functionmain
()
pholcus/socket/PhoSocketTest.go:122
Functionmain
()
pholcus/socket/PhoSocket.go:123
FunctionnoCheckRedirect
IsRedirectFunc的一个分支函数,防止302跳转,获取lostion
pholcus/keeper/login.go:49
Methodsend
(string)
reporter/reporter.go:6
← previous201–266 of 266, ranked by callers