MCPcopy Create free account
hub / github.com/awslabs/gap-text2sql / preprocess

Method preprocess

rat-sql-gap/seq2struct/commands/preprocess.py:20–28  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

18 config['model'])
19
20 def preprocess(self):
21 self.model_preproc.clear_items()
22 for section in self.config['data']:
23 data = registry.construct('dataset', self.config['data'][section])
24 for item in tqdm.tqdm(data, desc=section, dynamic_ncols=True):
25 to_add, validation_info = self.model_preproc.validate_item(item, section)
26 if to_add:
27 self.model_preproc.add_item(item, section, validation_info)
28 self.model_preproc.save()
29
30def add_parser():
31 parser = argparse.ArgumentParser()

Callers 1

mainFunction · 0.95

Calls 4

clear_itemsMethod · 0.45
validate_itemMethod · 0.45
add_itemMethod · 0.45
saveMethod · 0.45

Tested by

no test coverage detected