MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / Scanner

Class Scanner

tensorflow/core/lib/strings/scanner.h:35–240  ·  view source on GitHub ↗

Scanner provides simplified string parsing, in which a string is parsed as a series of scanning calls (e.g. One, Any, Many, OneLiteral, Eos), and then finally GetResult is called. If GetResult returns true, then it also returns the remaining characters and any captured substring. The range to capture can be controlled with RestartCapture and StopCapture; by default, all processed characters are c

Source from the content-addressed store, hash-verified

33// The range to capture can be controlled with RestartCapture and StopCapture;
34// by default, all processed characters are captured.
35class Scanner {
36 public:
37 // Classes of characters. Each enum name is to be read as the union of the
38 // parts - e.g., class LETTER_DIGIT means the class includes all letters and
39 // all digits.
40 //
41 // LETTER means ascii letter a-zA-Z.
42 // DIGIT means ascii digit: 0-9.
43 enum CharClass {
44 // NOTE: When adding a new CharClass, update the AllCharClasses ScannerTest
45 // in scanner_test.cc
46 ALL,
47 DIGIT,
48 LETTER,
49 LETTER_DIGIT,
50 LETTER_DIGIT_DASH_UNDERSCORE,
51 LETTER_DIGIT_DASH_DOT_SLASH, // SLASH is / only, not backslash
52 LETTER_DIGIT_DASH_DOT_SLASH_UNDERSCORE, // SLASH is / only, not backslash
53 LETTER_DIGIT_DOT,
54 LETTER_DIGIT_DOT_PLUS_MINUS,
55 LETTER_DIGIT_DOT_UNDERSCORE,
56 LETTER_DIGIT_UNDERSCORE,
57 LOWERLETTER,
58 LOWERLETTER_DIGIT,
59 LOWERLETTER_DIGIT_UNDERSCORE,
60 NON_ZERO_DIGIT,
61 SPACE,
62 UPPERLETTER,
63 RANGLE,
64 };
65
66 explicit Scanner(StringPiece source) : cur_(source) { RestartCapture(); }
67
68 // Consume the next character of the given class from input. If the next
69 // character is not in the class, then GetResult will ultimately return false.
70 Scanner& One(CharClass clz) {
71 if (cur_.empty() || !Matches(clz, cur_[0])) {
72 return Error();
73 }
74 cur_.remove_prefix(1);
75 return *this;
76 }
77
78 // Consume the next s.size() characters of the input, if they match <s>. If
79 // they don't match <s>, this is a no-op.
80 Scanner& ZeroOrOneLiteral(StringPiece s) {
81 str_util::ConsumePrefix(&cur_, s);
82 return *this;
83 }
84
85 // Consume the next s.size() characters of the input, if they match <s>. If
86 // they don't match <s>, then GetResult will ultimately return false.
87 Scanner& OneLiteral(StringPiece s) {
88 if (!str_util::ConsumePrefix(&cur_, s)) {
89 error_ = true;
90 }
91 return *this;
92 }

Callers 15

ParseTransformParametersFunction · 0.85
ConsumeAttrNameFunction · 0.85
ConsumeListPrefixFunction · 0.85
ConsumeQuotedStringFunction · 0.85
ConsumeAttrTypeFunction · 0.85
ConsumeInOutNameFunction · 0.85
ConsumeInOutRefOpenFunction · 0.85
ConsumeInOutRefCloseFunction · 0.85
ConsumeInOutNameOrTypeFunction · 0.85
ConsumeInOutTimesTypeFunction · 0.85
ConsumeControlOutNameFunction · 0.85
ConsumeDocNameColonFunction · 0.85

Calls 6

ErrorFunction · 0.50
ConsumePrefixFunction · 0.50
OneClass · 0.50
AnyFunction · 0.50
emptyMethod · 0.45
dataMethod · 0.45

Tested by 2

TEST_FFunction · 0.68
TESTFunction · 0.68