MCPcopy Create free account
hub / github.com/alibaba/zvec / insert_docs

Method insert_docs

tests/db/sqlengine/perf_bench_test.cc:380–406  ·  view source on GitHub ↗

Generates DocCount() documents with 8 tokens drawn from a 256-word vocabulary (each word hits ~3% of docs), plus a 1% word ("mediumword") and a 0.1% word ("rareword") for selectivity sweeps.

Source from the content-addressed store, hash-verified

378 // vocabulary (each word hits ~3% of docs), plus a 1% word ("mediumword")
379 // and a 0.1% word ("rareword") for selectivity sweeps.
380 static void insert_docs(const Segment::Ptr &segment) {
381 for (int64_t doc_id = 0; doc_id < DocCount(); ++doc_id) {
382 std::string content;
383 for (int k = 0; k < 8; ++k) {
384 if (k > 0) content += ' ';
385 content += "w" + std::to_string((doc_id + k * 31) % 256);
386 }
387 if (doc_id % 100 == 0) {
388 content += " mediumword";
389 }
390 if (doc_id % 1000 == 0) {
391 content += " rareword";
392 }
393
394 Doc doc;
395 doc.set_pk("pk_" + std::to_string(doc_id));
396 doc.set_doc_id(static_cast<uint64_t>(doc_id));
397 doc.set<std::string>("content", content);
398 doc.set<int32_t>("tag", static_cast<int32_t>(doc_id % 2));
399 auto status = segment->insert(doc);
400 if (!status.ok()) {
401 std::printf("[PERF] fts insert failed at doc %lld: %s\n",
402 static_cast<long long>(doc_id), status.c_str());
403 return;
404 }
405 }
406 }
407
408 protected:
409 static inline std::string seg_path_ = "./fts_perf_bench_collection";

Callers

nothing calls this directly

Calls 5

DocCountFunction · 0.85
c_strMethod · 0.80
to_stringFunction · 0.50
insertMethod · 0.45
okMethod · 0.45

Tested by

no test coverage detected