MCPcopy Create free account
hub / github.com/OpenRaiser/PaperFlow / dedupe_strings

Function dedupe_strings

experiments/baselines/knowledge_entity/runner.py:287–299  ·  view source on GitHub ↗
(values: Iterable[Any])

Source from the content-addressed store, hash-verified

285
286
287def dedupe_strings(values: Iterable[Any]) -> List[str]:
288 result: List[str] = []
289 seen: set[str] = set()
290 for value in values:
291 cleaned = str(value or "").strip()
292 if not cleaned:
293 continue
294 normalized = normalize_phrase_text(cleaned)
295 if not normalized or normalized in seen:
296 continue
297 seen.add(normalized)
298 result.append(cleaned)
299 return result
300
301
302def citation_count(row: Dict[str, Any]) -> Optional[float]:

Callers 3

metadata_entitiesFunction · 0.70
extract_entitiesFunction · 0.70
build_initial_profileFunction · 0.70

Calls 1

normalize_phrase_textFunction · 0.70

Tested by

no test coverage detected