UTF-32

UTF-16

@brief access the first element @sa https://json.nlohmann.me/api/basic_json/front/

@brief returns the allocator associated with the container @sa https://json.nlohmann.me/api/basic_json/get_allocator/

* @details get available customizable token max size, set by config[avail_token_size] * @return always equal to <avail_token_size - 2> */

* @details get <|startoftext|> <|endoftext|> etc. marks multiply, * set by config[major_boundary_factor] * @return <|startoftext|> index in

! @brief Parses a C-style string from the BSON input. @param[in,out] result A reference to the string variable where the read

! @param[in] len the length of the array or static_cast<std::size_t>(-1) for an array of indefinite size @param[in] tag_h

! @brief reads a CBOR byte array This function first reads starting bytes to determine the expected byte array length and then copies thi

! @param[in] len the length of the object or static_cast<std::size_t>(-1) for an object of indefinite size @param[in] tag

! @brief reads a CBOR string This function first reads starting bytes to determine the expected string length and then copies this number

! @brief get codepoint from 4 hex characters following `\u` For input "\u c1 c2 c3 c4" the codepoint is: (c1 * 0x1000) + (c2 * 0x0100)

return the locale-dependent decimal point

* @details get <|endoftext|> index in dictionary, set by config[tokenizer_dictionary_at] * @return <|endoftext|> index in dictionary */

return syntax error message

! @return character read from the input after ignoring all 'N' entries */

python line 135 of scheduling_lms_discrete.py

! @param[in] len the length of the array @return whether array creation completed */

! @brief reads a MessagePack byte array This function first reads starting bytes to determine the expected byte array length and then cop

! @param[in] len the length of the object @return whether object creation completed */

! @brief reads a MessagePack string This function first reads starting bytes to determine the expected string length and then copies this

return floating-point value

* @details get token index for padding, its different defined by mode * Examples: * SD - Using <|endoftext|> = 49407 as pad_token_idx

return position of last read token

* @details get <|startoftext|> index in dictionary, set by config[tokenizer_dictionary_at] * @return <|startoftext|> index in dictionary */

get next token from lexer

! @return whether array creation completed */

! @param[out] dim an integer vector storing the ND array dimensions @return whether reading ND array size vector is successful */

! @return whether object creation completed */

! @brief determine the type and size for a container In the optimized UBJSON format, a type and a size can be provided to allow for a mor

! @param[out] result determined size @param[in,out] is_ndarray for input, `true` means already inside an ndarray vector

! @brief reads a UBJSON string This function is either called after reading the 'S' byte explicitly indicating a string, or in case of an

! @param prefix the previously read or set type prefix @return whether value creation completed */

@brief return whether the value has a subtype @sa https://json.nlohmann.me/api/byte_container_with_subtype/has_subtype/

@brief return whether value is discarded @sa https://json.nlohmann.me/api/basic_json/is_discarded/

@brief return whether type is primitive @sa https://json.nlohmann.me/api/basic_json/is_primitive/

@brief helper to access iterator member functions in range-based for @sa https://json.nlohmann.me/api/basic_json/items/

@brief create a new JSON pointer by appending the right JSON pointer at the end of the left JSON pointer @sa https://json.nlohmann.me/api/json_pointer

create reverse iterator from iterator

! @param[in,out] r reference to a JSON value that is manipulated while parsing @param[in] allow_exceptions_ whether p

default constructor (for null values)