Preprocessing to normalize data items on a 0-1 scale.

A function for cleaning up the input text. Ideally we'd strip things like punctuation here, but instead we're just being lazy and lowercasing

Approximates the bandwidth (number of rows in each band) needed to get threshold. Threshold t = (1/b) ** (1/r) where

Approximates the bandwidth (number of rows in each band) needed to get threshold. Threshold t = (1/b) ** (1/r) where

Generate hashvals for this signature

Generate hashvals for this signature

Returns dim different hash functions

Returns dim different hash functions

Return a list of each disjoint set

Return a list of each disjoint set

Not using a generator here, unlike the initial implementation, both because it doesn't save a ton of memory in this use case and because

Return the signature for object s

Return the signature for object s

Find the sets containing the objects and merge them all.

Find the sets containing the objects and merge them all.

Find and return the name of the set containing the object.

Find and return the name of the set containing the object.

Create a new empty union-find structure.

Create a new empty union-find structure.

Iterate through all items ever found or unioned by this structure.

Iterate through all items ever found or unioned by this structure.

Return dim different hash functions

Return dim different hash functions

Jaccard similarity between two sets

Jaccard similarity between two sets

Not using a generator here, unlike the initial implementation, both because it doesn't save a ton of memory in this use case and because

Returns minhash signature for set s

Returns minhash signature for set s