MCPcopy Create free account
hub / github.com/apache/nutch / parse

Method parse

src/java/org/apache/nutch/parse/ParseUtil.java:138–156  ·  view source on GitHub ↗

Performs a parse by iterating through a List of preferred Parsers until a successful parse is performed and a Parse object is returned. If the parse is unsuccessful, a message is logged to the WARNING level, and an empty parse is returned. @throws ParserNotFound

(String url, WebPage page)

Source from the content-addressed store, hash-verified

136 * If there is an error parsing.
137 */
138 public Parse parse(String url, WebPage page) throws ParseException {
139 Parser[] parsers = null;
140 Parse parse = null;
141
142 String contentType = TableUtil.toString(page.getContentType());
143 parsers = this.parserFactory.getParsers(contentType, url);
144
145 for (int i = 0; i < parsers.length; i++) {
146 parse = parse(url, page, parsers[i]);
147 if (parse != null && ParseStatusUtils.isSuccess(parse.getParseStatus())) {
148 return parse;
149 }
150 }
151
152 LOG.warn("Unable to successfully parse content " + url + " of type "
153 + contentType);
154 return ParseStatusUtils.getEmptyParse(new ParseException(
155 "Unable to successfully parse content"), null);
156 }
157
158 private Parse parse(String url, WebPage page, Parser parser) {
159 if (LOG.isDebugEnabled()) {

Callers 4

parseMetaTagsMethod · 0.95
testMetaHTMLParsingMethod · 0.95
testJsoupHtmlParserMethod · 0.95
processMethod · 0.95

Calls 8

toStringMethod · 0.95
isSuccessMethod · 0.95
getParseStatusMethod · 0.95
getEmptyParseMethod · 0.95
runParserMethod · 0.95
getParsersMethod · 0.80
getParseMethod · 0.65
getContentTypeMethod · 0.45

Tested by 3

parseMetaTagsMethod · 0.76
testMetaHTMLParsingMethod · 0.76
testJsoupHtmlParserMethod · 0.76