MCPcopy Create free account
hub / github.com/davisking/dlib / test_with_tfidf

Function test_with_tfidf

dlib/test/feature_hashing.cpp:116–132  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

114// ----------------------------------------------------------------------------------------
115
116 void test_with_tfidf()
117 {
118 // The intended use: hashed counts in, TF-IDF weighted vectors out.
119 const std::vector<std::vector<std::string>> documents = {
120 {"the", "cat", "sat"},
121 {"the", "dog", "sat"},
122 {"the", "cat", "ran"}
123 };
124 std::vector<sparse_type> counts;
125 for (const auto& document : documents)
126 counts.push_back(hash_word_ngrams(document, 1, 1, 1 << 20));
127
128 tfidf_transformer<sparse_type> tfidf;
129 tfidf.train(counts);
130 DLIB_TEST(tfidf.vocabulary_size() == 5);
131 DLIB_TEST(std::abs(length(tfidf(counts[0])) - 1) < 1e-12);
132 }
133
134// ----------------------------------------------------------------------------------------
135

Callers 1

perform_testMethod · 0.85

Calls 6

hash_word_ngramsFunction · 0.85
absFunction · 0.85
vocabulary_sizeMethod · 0.80
lengthFunction · 0.50
push_backMethod · 0.45
trainMethod · 0.45

Tested by

no test coverage detected