The subset of `words` that appear in the dictionary of WORDS.

All edits that are one edit away from `word`.

Return the case-function appropriate for text: upper, lower, title, or just str.

remove ngrams with count less than mincount avoid dict comprehension as it creates a new temp dict and overloads the memory Args:

Most probable spelling correction for word.

Count the word statistics of a file :param desc: :param filename: :param countkeeper: :return:

All edits that are two edits away from `word`.

Args: m (): tag (): mode (): Returns:

Combine first and rem results into one (probability, words) pair :param first: a tuple in the form: probability, word :param

Conditional probability of word, given previous word if bigram is not in our list, then fall back to unigrams Args:

Spell-correct word in match, and preserve proper upper/lower/title case.

Generate possible spelling corrections for word.

Takes as input a list of words and returns the positions (indices) of the words that are in the context of a negation. :param list doc:

Return (log P(words), words), where words is the best estimated segmentation :param text: the text to be segmented :param pre

:param m: :return:

Break a string to its constituent words (using Viterbi algorithm)

return the sorted set so mathes random combinations of puncts will be mapped to the same token "!??!?!!", "?!!!!?!", "!!?", "

Read key,value pairs from file.

Estimate the sentiment polarity of a tokenized document. Args: doc (): a list of words (strings) neg_comma (): if True, the n

Remove tags from sentence

Return a list of all possible (first, rem) pairs with max length of first <=L :param text: :return:

extract words from text :param text: :return:

Replace *English* contractions in ``text`` str with their unshortened forms. N.B. The "'d" and "'s" forms are ambiguous (had/would, is/has/po

Probability of `word`.

Kwargs: omit (list): choose what tokens that you want to omit from the text. possible values: ['email', 'percent'

:param corpus: the statistics from which corpus to use for the spell correction.

Args: pipeline (list): list of terms to use for tokenization. Each term, is a key from the dict of regexes `expre

Args: lowercase (bool): set to True in order to lowercase the text verbose (bool): set to True to print each text af

Args: corpus (str): the statistics from which corpus to use for the spell correction. max_split_lengt

Spell-correct word in match, and preserve proper upper/lower/title case.

Correct all the words within a text, returning the corrected text.

Given a list of words and a set of word positions, mark the words in those positions. :param list doc: a list of words (strings) :param s

Return the product of a sequence of numbers.

Estimate the probability of an unknown word, penalizing its length :param key: the word :param total: the count of all tokens