Method generator

join/LSH/datasketch/minhash.py:320–350

Compute MinHashes in a generator. This method avoids unnecessary overhead when initializing many minhashes by reusing the initialized state. Args: b (Iterable): An Iterable of lists of bytes, each list is hashed in to one MinHash in the output.

(cls, b, **minhash_kwargs)

    @classmethod
    def generator(cls, b, **minhash_kwargs):
        '''Compute MinHashes in a generator. This method avoids unnecessary
        overhead when initializing many minhashes by reusing the initialized
        state.

        Args:
            b (Iterable): An Iterable of lists of bytes, each list is
                hashed in to one MinHash in the output.
            minhash_kwargs: Keyword arguments used to initialize MinHash,
                will be used for all minhashes.

        Returns:
            A generator of computed MinHashes.

        Example:

            .. code-block:: python

                from datasketch import MinHash
                data = [[b'token1', b'token2', b'token3'],
                        [b'token4', b'token5', b'token6']]
                for minhash in MinHash.generator(data, num_perm=64):
                    # do something useful
                    minhash

        '''
        m = cls(**minhash_kwargs)
        for _b in b:
            _m = m.copy()
            _m.update_batch(_b)
            yield _m