MCPcopy Create free account
hub / github.com/BIT-DataLab/LakeBench / update_batch

Method update_batch

join/LSH/datasketch/minhash.py:155–175  ·  view source on GitHub ↗

Update this MinHash with new values. The values will be hashed using the hash function specified by the `hashfunc` argument in the constructor. Args: b (list): List of values to be hashed using the hash function specified. Example: To update

(self, b)

Source from the content-addressed store, hash-verified

153 self.hashvalues = np.minimum(phv, self.hashvalues)
154
155 def update_batch(self, b):
156 '''Update this MinHash with new values.
157 The values will be hashed using the hash function specified by
158 the `hashfunc` argument in the constructor.
159
160 Args:
161 b (list): List of values to be hashed using the hash function specified.
162
163 Example:
164 To update with new string values (using the default SHA1 hash
165 function, which requires bytes as input):
166
167 .. code-block:: python
168
169 minhash = Minhash()
170 minhash.update_batch([s.encode('utf-8') for s in ["token1", "token2"]])
171 '''
172 hv = np.array([self.hashfunc(_b) for _b in b], dtype=np.uint64)
173 a, b = self.permutations
174 phv = np.bitwise_and(((hv * np.tile(a, (len(hv), 1)).T).T + b) % _mersenne_prime, _max_hash)
175 self.hashvalues = np.vstack([phv, self.hashvalues]).min(axis=0)
176
177 def jaccard(self, other):
178 '''Estimate the `Jaccard similarity`_ (resemblance) between the sets

Callers 7

minhash_multi_processFunction · 0.95
query_lshFunction · 0.95
minhash_LshFunction · 0.95
query_lshFunction · 0.95
minhash_multi_processFunction · 0.95
query_lshFunction · 0.95
generatorMethod · 0.80

Calls

no outgoing calls

Tested by

no test coverage detected