MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / ConsumeToken

Method ConsumeToken

tensorflow/lite/testing/message.cc:36–79  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

34 }
35
36 void ConsumeToken(std::string* token) override {
37 if (!valid_) return;
38 Message* current_node = nodes_.top();
39 if (*token == "{") {
40 // This is the beginning of a new message, names after the previous token.
41 if (previous_token_.empty()) {
42 valid_ = false;
43 return;
44 }
45 nodes_.push(current_node ? current_node->AddChild(previous_token_)
46 : nullptr);
47 previous_token_.clear();
48 } else if (*token == "}") {
49 // A message is being completed. There should be no previous token. Note
50 // that the top-level message never closes, so we should always have at
51 // least one entry in the stack.
52 if (nodes_.size() == 1 || !previous_token_.empty()) {
53 valid_ = false;
54 return;
55 }
56 if (current_node) {
57 current_node->Finish();
58 }
59 nodes_.pop();
60 } else if (*token == ":") {
61 // We reached the end of the 'key' portion of a field. Store the token
62 // until we have the 'value' portion.
63 if (previous_token_.empty()) {
64 valid_ = false;
65 return;
66 }
67 } else {
68 if (previous_token_.empty()) {
69 previous_token_.swap(*token);
70 } else {
71 // This is the 'value' portion of a field. The previous token is the
72 // 'key'.
73 if (current_node) {
74 current_node->SetField(previous_token_, *token);
75 }
76 previous_token_.clear();
77 }
78 }
79 }
80
81 bool valid() const { return valid_; }
82

Callers 1

TokenizeFunction · 0.45

Calls 9

emptyMethod · 0.45
pushMethod · 0.45
AddChildMethod · 0.45
clearMethod · 0.45
sizeMethod · 0.45
FinishMethod · 0.45
popMethod · 0.45
swapMethod · 0.45
SetFieldMethod · 0.45

Tested by

no test coverage detected