Function common_tokens_to_str

subprojects/llama.cpp/common/ngram-map.cpp:23–34

Print the values of a sublist of `llama_tokens & inp` to a string in the form [v0, v1, v2, ...].

// Print the values of a sublist of `llama_tokens & inp` to a string in the form [v0, v1, v2, ...].
static std::string common_tokens_to_str(const llama_tokens & inp, size_t start, size_t length) {
    std::ostringstream oss;
    oss << '[';
    for (size_t i = 0; i < length; ++i) {
        if (i > 0) {
            oss << ", ";
        }
        oss << inp[start + i];
    }
    oss << ']';
    return oss.str();
}


// n-gram simple