Returns position of a word in the vocabulary; if the word is not found, returns -1

* ======== GetWordHash ======== * Returns hash value of a word. The hash is an integer between 0 and * vocab_hash_size (default is 30E6). * * For

* ======== GetWordHash ======== * Returns hash value of a word. The hash is an integer between 0 and * vocab_hash_size (default is 30E6). * * For

* ======== AddWordToVocab ======== * Adds a new word to the vocabulary (one that hasn't been seen yet). */

* ======== AddWordToVocab ======== * Adds a new word to the vocabulary (one that hasn't been seen yet). */

* ======== ReadWord ======== * Reads a single word from a file, assuming space + tab + EOL to be word * boundaries. * * Parameters: * word - A

* ======== ReadWord ======== * Reads a single word from a file, assuming space + tab + EOL to be word * boundaries. * * NOTE: This function is id

* ======== SearchVocab ======== * Lookup the index in the 'vocab' table of the given 'word'. * Returns -1 if the word is not found. * This function

* ======== SortVocab ======== * Sorts the vocabulary by frequency using word counts, and removes words that * occur fewer than 'min_count' times in

* ======== CreateBinaryTree ======== * Create binary Huffman tree using the word counts. * Frequent words will have short unique binary codes. * Hu

* ======== InitNet ======== * */

* ======== InitUnigramTable ======== * This table is used to implement negative sampling. * Each word is given a weight equal to it's frequency (wor

* ======== LearnVocabFromTrainFile ======== * Builds a vocabulary from the words found in the training file. * * This function will also build a ha

* ======== LearnVocabFromTrainFile ======== * Builds a vocabulary from the words found in the training file. * * This function will also build a ha

* ======== ReadWordIndex ======== * Reads the next word from the training file, and returns its index into the * 'vocab' table. */

Reduces the vocabulary by removing infrequent tokens

Reduces the vocabulary by removing infrequent tokens

* ======== SortVocab ======== * Sorts the vocabulary by frequency using word counts, and removes words that * occur fewer than 'min_count' times in

* ======== TrainModel ======== * Main entry point to the training process. */

* ======== TrainModel ======== * Main body of this tool. * * This function performs the phrase detection and simultaneously generates a * new tra

Reads a word and returns its index in the vocabulary

* ======== TrainModelThread ======== * This function performs the training of the model. */

Used later for sorting by word counts

Used later for sorting by word counts