return the index of cluster closest to the input

finds the aggregate distance between elements of cluster1 and elements of cluster2

training set consists of parts (meesage, is_spam)

This method splits a dataset into k folds

returns the result of subtracting from every value in A the mean value of its column. the resulting matrix has mean 0 in every column

given a list of class probabilities, compute the entropy

use gradient descent to fit a ridge regression with penalty alpha

This method evaluates the algorithm using a cross validation split

the sum of the lengths of the shortest paths to each other user

This method calculates the gini index for a split dataset

returns a defaultdict(list), where each input item is in the list whose key is key_fn(item)

generator that returns the elements of data in random order

This method loads a csv file

the gradient of the log likelihood corresponding to the i-th data point

here i is the index of the data point, j the index of the derivative

assumes that labels are ordered from nearest to farthest

buckets the points and counts how many in each bucket

finds the 'middle-most' value of v

returns a list, might be more than one mode

returns the z for which P(Z >= z) = probability

the fraction of words in document 'd' that are assigned to 'topic' (plus some smoothing)

the fraction of words in document 'd' that are assigned to 'topic' (plus some smoothing)

given a list of parsers (some of which may be None) apply the appropriate one to each element of the input_row

wrap a reader to apply the parsers to each of its rows

find the entropy from this partition of data into subsets

turn a list of dicts into the list of field_name values

This method makes a prediction with a decision tree

return the projection of v onto w

the fraction of variation in y captured by the model

using the 5% significance levels

for each row of X projects the row onto w, and subtracts the result from the row

projects v onto w and subtracts the result from v

gradient of just the ridge penalty

flip a fair coin 1000 times, True = heads, False = tails

define a new function that wraps f and return it

returns i with probability weights[i] / sum(weights)

assigns word probabilities to messages

split data into fractions [prob, 1 - prob]

split data into fractions [prob, 1 - prob]

the gradient corresponding to the ith squared error term

This method converts a string column to int

This method converts a string column to float

This method creates a random subsample from the dataset with replacement

This method split a dataset based on an attribute and an attribute value

given a document and a word in that document, return the weight for the k-th topic

The total squared variation of y_i's from their mean

The total squared variation of y_i's from their mean

try to parse value using the appropriate function from parser_dict

adds two vectors componentwise