MCPcopy Create free account
hub / github.com/BIT-DataLab/LakeBench / index_old

Method index_old

join/LSH/datasketch/lshensemble.py:189–225  ·  view source on GitHub ↗

给出所有集合的键、MinHashes和大小的索引。 它只能在索引创建后被调用一次。 Args: entries (`iterable` of `tuple`): 需要格式为 `(key, minhash, size)`, 其中key是一个集合的唯一标识符,minhash是该集合的MinHash,size是该集合的大小。 Note: size需要为正数

(self, entries)

Source from the content-addressed store, hash-verified

187 self.indexes[part][r].hashtables[i].insert(j, lshE.indexes[part][r].hashtables[i][j])
188
189 def index_old(self, entries):
190 '''
191 给出所有集合的键、MinHashes和大小的索引。
192 它只能在索引创建后被调用一次。
193
194 Args:
195 entries (`iterable` of `tuple`): 需要格式为 `(key, minhash, size)`,
196 其中key是一个集合的唯一标识符,minhash是该集合的MinHash,size是该集合的大小。
197
198 Note:
199 size需要为正数
200 '''
201 if not self.is_empty():
202 raise ValueError("Cannot call index again on a non-empty index")
203 if not isinstance(entries, list):
204 queue = deque([])
205 for key, minhash, size in entries:
206 if size <= 0:
207 raise ValueError("Set size must be positive")
208 queue.append((key, minhash, size))
209 entries = list(queue)
210 if len(entries) == 0:
211 raise ValueError("entries is empty")
212 # 创建最优的分区
213 sizes, counts = np.array(sorted(
214 Counter(e[2] for e in entries).most_common())).T
215 partitions = optimal_partitions(sizes, counts, len(self.indexes))
216 for i, (lower, upper) in enumerate(partitions):
217 self.lowers[i], self.uppers[i] = lower, upper
218 # 将候选索引插入到对应分区
219 entries.sort(key=lambda e : e[2])
220 curr_part = 0
221 for key, minhash, size in entries:
222 if size > self.uppers[curr_part]:
223 curr_part += 1
224 for r in self.indexes[curr_part]:
225 self.indexes[curr_part][r].insert(key, minhash)
226
227 def query(self, minhash, size):
228 ''&#x27;

Callers

nothing calls this directly

Calls 3

is_emptyMethod · 0.95
optimal_partitionsFunction · 0.90
insertMethod · 0.45

Tested by

no test coverage detected