MCPcopy Create free account
hub / github.com/pytorch/examples / add_word

Method add_word

word_language_model/data.py:10–14  ·  view source on GitHub ↗
(self, word)

Source from the content-addressed store, hash-verified

8 self.idx2word = []
9
10 def add_word(self, word):
11 if word not in self.word2idx:
12 self.idx2word.append(word)
13 self.word2idx[word] = len(self.idx2word) - 1
14 return self.word2idx[word]
15
16 def __len__(self):
17 return len(self.idx2word)

Callers 1

tokenizeMethod · 0.80

Calls

no outgoing calls

Tested by

no test coverage detected