MCPcopy Index your code
hub / github.com/RustPython/RustPython / _guess_quote_and_delimiter

Method _guess_quote_and_delimiter

Lib/csv.py:272–346  ·  view source on GitHub ↗

Looks for text enclosed between two identical quotes (the probable quotechar) which are preceded and followed by the same character (the probable delimiter). For example: ,'some text', The quote with the most wins, same with the delim

(self, data, delimiters)

Source from the content-addressed store, hash-verified

270
271
272 def _guess_quote_and_delimiter(self, data, delimiters):
273 """
274 Looks for text enclosed between two identical quotes
275 (the probable quotechar) which are preceded and followed
276 by the same character (the probable delimiter).
277 For example:
278 ,'some text',
279 The quote with the most wins, same with the delimiter.
280 If there is no quotechar the delimiter can't be determined
281 this way.
282 """
283 import re
284
285 matches = []
286 for restr in (r'(?P<delim>[^\w\n"\'])(?P<space> ?)(?P<quote>["\']).*?(?P=quote)(?P=delim)', # ,".*?",
287 r'(?:^|\n)(?P<quote>["\']).*?(?P=quote)(?P<delim>[^\w\n"\'])(?P<space> ?)', # ".*?",
288 r'(?P<delim>[^\w\n"\'])(?P<space> ?)(?P<quote>["\']).*?(?P=quote)(?:$|\n)', # ,".*?"
289 r'(?:^|\n)(?P<quote>["\']).*?(?P=quote)(?:$|\n)&#x27;): # ".*?" (no delim, no space)
290 regexp = re.compile(restr, re.DOTALL | re.MULTILINE)
291 matches = regexp.findall(data)
292 if matches:
293 break
294
295 if not matches:
296 # (quotechar, doublequote, delimiter, skipinitialspace)
297 return ('', False, None, 0)
298 quotes = {}
299 delims = {}
300 spaces = 0
301 groupindex = regexp.groupindex
302 for m in matches:
303 n = groupindex['quote'] - 1
304 key = m[n]
305 if key:
306 quotes[key] = quotes.get(key, 0) + 1
307 try:
308 n = groupindex['delim'] - 1
309 key = m[n]
310 except KeyError:
311 continue
312 if key and (delimiters is None or key in delimiters):
313 delims[key] = delims.get(key, 0) + 1
314 try:
315 n = groupindex['space'] - 1
316 except KeyError:
317 continue
318 if m[n]:
319 spaces += 1
320
321 quotechar = max(quotes, key=quotes.get)
322
323 if delims:
324 delim = max(delims, key=delims.get)
325 skipinitialspace = delims[delim] == spaces
326 if delim == '\n': # most likely a file with a single column
327 delim = ''
328 else:
329 # there is *no* delimiter, it's a single column of quoted data

Callers 1

sniffMethod · 0.95

Calls 6

maxFunction · 0.85
escapeMethod · 0.80
compileMethod · 0.45
findallMethod · 0.45
getMethod · 0.45
searchMethod · 0.45

Tested by

no test coverage detected