Load given list of text files using the `load_func` function. Args: file_paths: list of file paths load_func: loading function Returns: flat list of instances
(file_paths: List[str], load_func=_load_kaggle_text_norm_file)
| 76 | |
| 77 | |
| 78 | def load_files(file_paths: List[str], load_func=_load_kaggle_text_norm_file) -> List[Instance]: |
| 79 | """ |
| 80 | Load given list of text files using the `load_func` function. |
| 81 | |
| 82 | Args: |
| 83 | file_paths: list of file paths |
| 84 | load_func: loading function |
| 85 | |
| 86 | Returns: flat list of instances |
| 87 | """ |
| 88 | res = [] |
| 89 | for file_path in file_paths: |
| 90 | res.extend(load_func(file_path=file_path)) |
| 91 | return res |
| 92 | |
| 93 | |
| 94 | def clean_generic(text: str) -> str: |
no outgoing calls
no test coverage detected
searching dependent graphs…