MCPcopy Create free account
hub / github.com/antlr/codebuff / computeFeatureVectorForToken

Method computeFeatureVectorForToken

src/org/antlr/codebuff/Trainer.java:247–263  ·  view source on GitHub ↗
(int i)

Source from the content-addressed store, hash-verified

245 }
246
247 public void computeFeatureVectorForToken(int i) {
248 Token curToken = tokens.get(i);
249 if ( curToken.getType()==Token.EOF ) return;
250
251 int[] features = getFeatures(i);
252
253 int injectNL_WS = getInjectWSCategory(tokens, i);
254
255 int aligned = -1; // "don't care"
256 if ( (injectNL_WS&0xFF)==CAT_INJECT_NL ) {
257 TerminalNode node = tokenToNodeMap.get(curToken);
258 aligned = getAlignmentCategory(doc, node, indentSize);
259 }
260
261 // track feature -> injectws, align decisions for token i
262 corpus.addExemplar(doc, features, injectNL_WS, aligned);
263 }
264
265 public static int getInjectWSCategory(CodeBuffTokenStream tokens, int i) {
266 int precedingNL = getPrecedingNL(tokens, i); // how many lines to inject

Callers 1

computeFeatureVectorsMethod · 0.95

Calls 6

getFeaturesMethod · 0.95
getInjectWSCategoryMethod · 0.95
getAlignmentCategoryMethod · 0.95
addExemplarMethod · 0.80
getMethod · 0.65
getTypeMethod · 0.45

Tested by

no test coverage detected