MCPcopy Create free account
hub / github.com/BIT-DataLab/LakeBench / preprocessListValues

Function preprocessListValues

union/TUS/generalFunctions.py:50–55  ·  view source on GitHub ↗
(valueList)

Source from the content-addressed store, hash-verified

48
49#removes punctuations and whitespaces from list items
50def preprocessListValues(valueList):
51 valueList = [x.lower() for x in valueList if checkIfNullString(x) !=0]
52 valueList = [re.sub(r'[^\w]', ' ', string) for string in valueList]
53 valueList = [x.replace('nbsp','') for x in valueList ] #remove html whitespace
54 valueList = [" ".join(x.split()) for x in valueList]
55 return valueList
56
57#checks different types of nulls in string
58def checkIfNullString(string):

Callers 1

expandQueryFunction · 0.70

Calls 1

checkIfNullStringFunction · 0.70

Tested by

no test coverage detected