Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/commoncrawl/cc-webgraph
/ functions
Functions
144 in github.com/commoncrawl/cc-webgraph
⨍
Functions
144
◇
Types & classes
14
↓ 20 callers
Method
testSorted
Test whether node names are properly sorted and IDs are correctly assigned (sequentially, strictly monotonically increasing, no gaps)
src/test/java/org/commoncrawl/webgraph/TestHostToDomainGraph.java:238
↓ 20 callers
Method
vertexLabelToId
(String label)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:114
↓ 15 callers
Method
nextInt
{@inheritDoc}
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:85
↓ 12 callers
Method
add
(long hostId)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:160
↓ 12 callers
Method
convert
(HostToDomainGraph converter, String[] hostGraph)
src/test/java/org/commoncrawl/webgraph/TestHostToDomainGraph.java:214
↓ 12 callers
Method
doCount
@param countHosts if true count the number of hosts per domain
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:266
↓ 12 callers
Method
hasNext
{@inheritDoc}
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:66
↓ 11 callers
Method
toString
()
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:50
↓ 8 callers
Method
print
(String s)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:267
↓ 7 callers
Method
vertexIdToLabel
(long id)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:102
↓ 6 callers
Method
compareTo
(Domain o)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:170
↓ 6 callers
Method
getCount
@return the count how often the last integer (returned by {@link nextInt()}) was seen in the input iterators
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:118
↓ 5 callers
Method
compareRevDomainsSafe
(String d1, String d2)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:192
↓ 5 callers
Method
getValue
(long id)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:334
↓ 5 callers
Method
reverseHost
Reverse host name, eg. <code>www.example.com</code> is reversed to <code>com.example.www</code>. Can also be used to "unreverse" a reversed host name.
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:320
↓ 5 callers
Method
showHelp
()
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:537
↓ 4 callers
Method
multiPartSuffixesAsDomains
@param include if true map host names equal to any multi-part public suffix (the suffix contains a dot) (eg. <code>gov.uk</code> or
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:299
↓ 4 callers
Method
successorIntStream
(String vertexLabel)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:172
↓ 4 callers
Method
successors
(long vertexId)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:160
↓ 3 callers
Method
next
{@inheritDoc} @deprecated Please use {@link nextInt()} instead.
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:76
↓ 3 callers
Method
nextPreferenceElement
()
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:42
↓ 3 callers
Method
outdegree
(long vertexId)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:144
↓ 3 callers
Method
setStripWww
@param stripWww if true the www. prefix is stripped
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:306
↓ 3 callers
Method
toString
()
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:165
↓ 2 callers
Method
assignRank
(int[] ranks, IntComparator comp)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:93
↓ 2 callers
Method
assignRank
(long[][] ranks, LongComparator comp)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:224
↓ 2 callers
Method
convert
(Function<String, String> func, Stream<String> in, PrintStream out)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:489
↓ 2 callers
Method
doPrivateDomains
@param privateDomains if true map host to domain names using also the suffixes from the <a href= "https://
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:277
↓ 2 callers
Method
finishNodes
(PrintStream out)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:501
↓ 2 callers
Method
frequencies
Count strings in a stream. Sort the resulting string-count pairs by decreasing count (frequency) and secondarily by string in lexicographic order. @p
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:305
↓ 2 callers
Method
getNodeLine
(Domain domain)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:436
↓ 2 callers
Method
getTopLevelDomain
(String reversedDomainName)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:379
↓ 2 callers
Method
getVertex
(String vertexLabel)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:83
↓ 2 callers
Method
intersect
Intersection of two sorted lists
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:388
↓ 2 callers
Method
logProgress
()
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:57
↓ 2 callers
Method
printVertices
(LazyIntIterator it)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:271
↓ 2 callers
Method
pwn
Print the current working node / vertex.
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:124
↓ 2 callers
Method
reverseDomainName
Reverse or "unreverse" a host/domain name: <code>com.example.www</code> is reversed to <code>www.example.com</code> and vice versa. @param domainName
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:498
↓ 2 callers
Method
setValue
(long id, long value)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:330
↓ 2 callers
Method
setVertex
(String vertexLabel)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:91
↓ 2 callers
Method
sharedSuccessors
(long[] vertices)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:330
↓ 2 callers
Method
showHelp
()
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:263
↓ 2 callers
Method
showHelp
()
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:158
↓ 2 callers
Method
successorStream
(String vertexLabel)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:168
↓ 2 callers
Method
successors
()
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:63
↓ 1 callers
Method
assignHarmonicCentralityRank
()
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:106
↓ 1 callers
Method
assignPageRankRank
()
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:110
↓ 1 callers
Method
convert
(Function<String, String> func, Stream<String> in, PrintStream out)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:130
↓ 1 callers
Method
convert
(Stream<String> in, DataOutputStream out)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:134
↓ 1 callers
Method
getHarmonicCentralityRank
(long id)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:118
↓ 1 callers
Method
getHarmonicCentralityValue
(long id)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:114
↓ 1 callers
Method
getNodeIDs
(String[] graph)
src/test/java/org/commoncrawl/webgraph/TestHostToDomainGraph.java:230
↓ 1 callers
Method
getNodeNames
(String[] graph)
src/test/java/org/commoncrawl/webgraph/TestHostToDomainGraph.java:226
↓ 1 callers
Method
getPageRankRank
(long id)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:126
↓ 1 callers
Method
getPageRankValue
(long id)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:122
↓ 1 callers
Method
getRegisteredDomain
Get the registered domain for a host name based on the ICANN section of the <a href="https://www.publicsuffix.org/">public suffix list</a>. @see Effe
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:459
↓ 1 callers
Method
indegree
(long vertexId)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:152
↓ 1 callers
Method
isSafeToOutput
Whether the domain is safe to output given the reversed domain name seen next. @param nextDomainRevName next name in lexicographically sorted list of
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:184
↓ 1 callers
Method
loadHarmonicCentrality
(String ranksHC)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:47
↓ 1 callers
Method
loadPageRank
(String ranksPR)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:52
↓ 1 callers
Method
predecessorIntersect
(long vertexId, long[] vertexIds)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:314
↓ 1 callers
Method
predecessors
(long vertexId)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:290
↓ 1 callers
Method
queueDomain
Add the domain name to the queue if it is not already queued. Flush the queue, assuming properly sorted input. @param sb domains which are sa
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:406
↓ 1 callers
Method
read
(Stream<String> in)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:99
↓ 1 callers
Method
readJoinNode
(String line)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:61
↓ 1 callers
Method
reportConfig
()
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:116
↓ 1 callers
Method
setPrefSet
(Stream<String> pref)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:52
↓ 1 callers
Method
sharedPredecessors
(long[] vertices)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:322
↓ 1 callers
Method
skip
{@inheritDoc}
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:125
↓ 1 callers
Method
stripCounts
(String[] domainGraph)
src/test/java/org/commoncrawl/webgraph/TestHostToDomainGraph.java:222
↓ 1 callers
Method
successorIntersect
(long vertexId, long[] vertexIds)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:204
↓ 1 callers
Method
successorTopLevelDomainCounts
(String vertexLabel)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:184
↓ 1 callers
Method
successorTopLevelDomainStream
(ImmutableGraph graph, long vertexId)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:208
↓ 1 callers
Method
validatePreferenceVector
Check preference vector whether values sum up to 1.0, see <a href= "https://law.di.unimi.it/software/law-docs/it/unimi/dsi/law/rank/SpectralRanking.ht
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:149
↓ 1 callers
Method
write
(DataOutputStream out)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:107
Method
CountingMergedIntIterator
@param iterators input iterators
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:54
Method
CreatePreferenceVector
(double defVal)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:38
Method
Domain
(String name, String revName, long id, long numberOfHosts)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:136
Method
Graph
(String name)
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:64
Method
GraphExplorer
(String name)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:75
Method
HostToDomainGraph
()
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:255
Method
HostToDomainGraphBig
(long maxSize)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:521
Method
QueuedIterator
(LazyIntIterator iterator)
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:29
Method
Vertex
(String label)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:40
Method
addRanks
(String line)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:134
Method
assignHarmonicCentralityRank
()
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:237
Method
assignPageRankRank
()
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:241
Method
cn
Change the current working node / vertex. @param vertexLabel vertex label (node name)
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:106
Method
compareHarmonicCentralityIndirect
(int k1, int k2)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:57
Method
compareHarmonicCentralityIndirect
(long k1, long k2)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:188
Method
comparePageRankIndirect
(int k1, int k2)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:73
Method
comparePageRankIndirect
(long k1, long k2)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:204
Method
compareTo
(QueuedIterator o)
src/main/java/org/commoncrawl/webgraph/CountingMergedIntIterator.java:34
Method
convertEdge
(String line)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:471
Method
convertNode
(String line)
src/main/java/org/commoncrawl/webgraph/HostToDomainGraph.java:338
Method
convertNode
(String line)
src/main/java/org/commoncrawl/webgraph/CreatePreferenceVector.java:92
Method
difference
Difference of two sorted lists: a \ b
src/main/java/org/commoncrawl/webgraph/explore/Graph.java:428
Method
getGraph
()
src/main/java/org/commoncrawl/webgraph/explore/GraphExplorer.java:79
Method
getHarmonicCentralityRank
(long id)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:249
Method
getHarmonicCentralityValue
(long id)
src/main/java/org/commoncrawl/webgraph/JoinSortRanks.java:245
next →
1–100 of 144, ranked by callers