Detects escaped newlines (line continuations) and signals them with the special '\r' character (that otherwise is not used). Keeps track of phys

Appends a character to the token buffer, always making sure there is room.

Undo action of a get() lookahead call. An attempt at undoing an EOF read has no effect. Since get() encodes logical line endings with \n and con

Make sure we don't attempt to recover from problems in subrules.

Sample size equals length of token sequence.

Outputs the token to open file fp. When do_quote is > 0, output token double-quoted, when do_quote is < 0, output token without double-quotes,

Checks whether (JSMN_STRING) token equals a certain string value. */

Checks whether token is of a certain type. */

Parse program arguments Parameters: - parser -- argument parser as an object of ArgumentParser to add arguments Ret

Reset system seeds for random generators

Make program arguments parser for machine learning for analysis of source code problems Parameters: - description -- string describi

Setup all seeds used in dataset construction Function should be executed only once before making all datasets Parameters:

Compute layer initializer Parameters: - seed -- seed for random generator used by analyzer If it is None

Adds a terminating NUL character which does not change the token length.

Make a stack of dense layers with classifier Classifier is either softmax for more than 2 classes or sigmoid for 2 classes

Phase 1. Forms logical lines by resolving escaped newlines (line continuations). */

Randomize list of ordered elements by shuffling it in place It is assumed that the elements are in order which is the same f

Set seed for current randomization procedure Parameters: - seed_name -- name of the seed

Make sequential convolutional DNN building block Parameters: - convolutions -- specification of convolutional layers

Get either the specified checkpoint or the latest one Parameters: - checkpoint_dir -- Directory with checkpoints - checkpoint -

Experomental version of function for making functional dnn model and compiling it Parameters: - inputs -- list o

Get unique seed

Either set this file's input language explicitly via a string or use the filename extension to determine the language. If neither works out, use

Make Siamese block of CNN Outputs are Max pooled tensors of convolved input sequence Parameters are taken from members of ob

Compute absolute value of difference of two tensor Parameters: - input1 -- 1-st operand - input2 -- 2-nd operand

Make a dataset for source code similarity analyser Parameters: - start_problem -- index of fist problem to use for sampl

Add arguments defining regularization Parameters: - parser -- argument parser as an object of ArgumentParser to add arg

Add arguments defining DNN processing sequences of tokens Parameters: - parser -- argument parser as an object of ArgumentParser

Check definition of convolution by program arguments Parameters: - args object with program arguments

Assuming there is at least one period which seperates the real name and the file extension return file_name and file_type @param fileName @return resu

Load tokenized samples of all solutions of programming problems, transform each sample into bag or sequence of tokens, Parti

- Create callback function for writing down checkpoints Parameter: - checkpoint_dir -- Directory for writing down checkpoints - moni

Make path to file and check its existance Check if the path exists and exit if it is not found The existance of the path is checked only

Set up writing down checkpoints: - Make checkpoint directory is it does not exist - Find latest checkpoint Parameter: - checkpoin

Write problem list to file to report test, train, validation problems Parameters: - problems -- list of problems

Make functional dnn model and compile it Parameters: - inputs -- list of dnn inputs - output -- dnn output

Check that all solutiosn of a given problem are in dataset Parameters: - problem -- problem to check - solutions -- set of solution

Compute loss function Parameters: - labels -- number of labels (classes)

Compile sequential DNN Parameters: - dnn -- DNN to compile - loss -- loss to use for training

Checks whether token is string or number literal. */

Get list of problems satisfying required criteria: - number of solutions is not less than threshold - number of problems is not more than

Generic binary search lookup in some keyword table. `word' to be searched must be NUL-terminated C string. `table' is array of const char * of `

Compute label distribution Parameters: - labels - list of sample labels - n_labels - number of labels

Loads tokenized samples of all solutions of programming problems. Transforms each sample into bag or sequence of tokens Actu

Make Dataset options: currently they are chard options for multi GPU mode Parameters: - policy -- policy to shard dataset:

Skips all tokens that form the current token type. Returns the token right after. */

Make similarity dataset in the form of TensorFlow Dataset of from_generator type The dataset has samples of all pairs of pr

Make datasets for training source code similarity analyser Both training and validation parts of dataset are made. The vali

Make datasets for training source code similarity analyser Both training and validation parts of dataset are made. The valid

Returns string representation for JSON type. */

Write report on similarity analysis

Undo JSON escaping: \\ => \. */

Escape token for output as XML text.

emb_dim (int): node embedding dimensionality

Add dot product classifier Dot product is not normalized but is transformed by sigmoid to probabilities as it helps converge

Check loaded dataset with respect to source dataset directory Parameters: - ds -- dictionary defining dataset to be checked