Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/Lichanglu/pholcus
/ functions
Functions
266 in github.com/Lichanglu/pholcus
⨍
Functions
266
◇
Types & classes
39
Method
LogInfo
LogError logs normal info.
common/mlog/filelog.go:91
Function
LogInst
LogInst get the singleton filelog object.
common/mlog/filelog.go:25
Function
MakeHash
string to hash
common/util/util.go:106
Function
MakeMd5
制作特征值方法二
common/util/util.go:126
Function
MakeUnique
制作特征值方法一
common/util/util.go:120
Method
MinSizeHint
()
pholcus/gui/logview.go:44
Function
New
()
pholcus/crawler/crawl.go:24
Function
New
()
common/deduplicate/deduplicate.go:16
Function
New
()
pipeline/pipeline.go:30
Function
NewCollector
()
pipeline/collector/collector.go:24
Function
NewConfig
()
common/config/config.go:18
Function
NewDataCell
(ruleName string, data map[string]string, url string, parentUrl string, downloadTime string)
pipeline/collector/datacell.go:6
Function
NewHttpDownloader
()
downloader/downloader_http.go:32
Function
NewQueue
(size int)
common/queue/queue.go:8
Function
NewRequest
NewRequest returns initialized Request object. The respType is json, jsonp, html, text
downloader/context/request.go:45
Function
NewResponse
NewResponse returns initialized Response object.
downloader/context/response.go:38
Method
Open
Open makes log open.
common/mlog/mlog.go:24
Function
PhoSocketAccept
* *@服务端用 * *PhoSocketAccept()为幽灵蛛socket同意连接函数 *ln为一个监听句柄 *输出类型为net.Conn,一个握手连接,下一步可以进行接收,发送 */
pholcus/socket/PhoSocketTest.go:58
Function
PhoSoketDial
* *@客户端用 * *PhoSoketDial()为幽灵蛛socket拨号函数,请求服务端 *PhoSocketServer为预定义常量:server:port *输出类型为net.Conn,一个握手连接,下一步可以进行接收,发送 */
pholcus/socket/PhoSocket.go:43
Function
PhoSoketLisent
建立连接 func 接受 func 发送 * *@服务端用 * *PhoSoketLisent()为幽灵蛛socket监听函数 *PhoSocketServer为预定义常量:server:port *输出类型为net.Listener,一个监听句柄 */
pholcus/socket/PhoSocketTest.go:28
Method
Printf
(format string, v ...interface{})
reporter/report.go:16
Method
Println
(v ...interface{})
reporter/report.go:21
Method
Println
Println output the str to os.Stderr.
common/mlog/strace.go:37
Method
Push
(req *context.Request)
scheduler/src_manage.go:33
Method
Push
(req *context.Request)
scheduler/scheduler.go:38
Method
Push
(req *context.Request)
pholcus/pholcus.go:19
Method
Receive
(req context.Request)
pholcus/pholcus.go:49
Method
ResetHtmlParser
GetHtmlParser returns goquery object binded to target crawl result.
downloader/context/response.go:125
Method
RowCount
Called by the TableView from SetModel and every time the model publishes a RowsReset event.
pholcus/gui/guispider.go:56
Function
Run
()
pholcus/gui/guimain.go:19
Method
SectionContent
(section string)
common/config/config.go:247
Method
SectionContents
()
common/config/config.go:251
Method
SectionGetDuration
(section string, key string)
common/config/config.go:232
Method
SectionGetSlice
(section string, key string, separator string)
common/config/config.go:236
Method
SectionHas
(section string, key string)
common/config/config.go:193
Method
SectionSet
(section string, key string, value string)
common/config/config.go:210
Method
Sections
()
common/config/config.go:182
Method
SetChecked
Called by the TableView when the user toggled the check box of a given row.
pholcus/gui/guispider.go:86
Method
SetOutsource
(serve bool)
pholcus/pholcus.go:37
Method
SetRequest
SetRequest saves request oject of self page.
downloader/context/response.go:92
Method
SetTemp
(key string, value interface{})
downloader/context/request.go:248
Method
SetUrl
(url string)
downloader/context/request.go:191
Method
ShowOutFeild
获取任意规则采集语义字段
spiders/spider/spider.go:73
Method
SizeHint
()
pholcus/gui/logview.go:48
Method
Sort
Called by the TableView to sort the model.
pholcus/gui/guispider.go:104
Method
Start
func NewSpider() *Spider { sp := new(Spider) sp.RuleTree = &RuleTree{ Nodes: make(map[string]*Rule), } return sp }
spiders/spider/spider.go:30
Method
Start
任务执行入口
pholcus/crawler/crawl.go:41
Method
Start
()
pipeline/pipeline.go:12
Method
Start
()
pipeline/pipeline.go:50
Function
StartConf
StartConf is used in Spider for initialization at first time.
common/etc_config/etc_config.go:31
Function
StraceInst
StraceInst get the singleton strace object.
common/mlog/strace.go:21
Method
StringWithMeta
()
common/config/config.go:102
Method
Swap
(i, j int)
pholcus/gui/guispider.go:144
Method
TryOutsource
()
downloader/context/request.go:231
Method
Use
(spiderId int)
scheduler/src_manage.go:39
Method
Value
Called by the TableView when it needs the text to display for a given cell.
pholcus/gui/guispider.go:61
Method
WndProc
(hwnd win.HWND, msg uint32, wParam, lParam uintptr)
pholcus/gui/logview.go:84
Function
XML2mapstr
simple xml to string support utf8
common/util/util.go:80
Function
init
()
reporter/report.go:28
Function
init
()
pholcus/pholcus.go:56
Function
init
()
config/config.go:55
Function
main
()
main.go:7
Function
main
()
pholcus/socket/PhoSocketTest.go:122
Function
main
()
pholcus/socket/PhoSocket.go:123
Function
noCheckRedirect
IsRedirectFunc的一个分支函数,防止302跳转,获取lostion
pholcus/keeper/login.go:49
Method
send
(string)
reporter/reporter.go:6
← previous
201–266 of 266, ranked by callers