MCPcopy
hub / github.com/open-compass/VLMEvalKit / prepare_tsv

Method prepare_tsv

vlmeval/dataset/image_base.py:84–104  ·  view source on GitHub ↗
(self, url, file_md5=None)

Source from the content-addressed store, hash-verified

82 return dict(self.data.iloc[idx])
83
84 def prepare_tsv(self, url, file_md5=None):
85 data_root = LMUDataRoot()
86 os.makedirs(data_root, exist_ok=True)
87 update_flag = False
88 file_name = url.split('/')[-1]
89 data_path = osp.join(data_root, file_name)
90 self.data_path=data_path
91 if osp.exists(data_path) and (file_md5 is None or md5(data_path) == file_md5):
92 pass
93 else:
94 warnings.warn('The dataset tsv is not downloaded')
95 download_file(url, data_path)
96 update_flag = True
97
98 if file_size(data_path, 'GB') > 1:
99 local_path = data_path.replace('.tsv', '_local.tsv')
100 if not osp.exists(local_path) or os.environ.get('FORCE_LOCAL', None) or update_flag:
101 from ..tools import LOCALIZE
102 LOCALIZE(data_path, local_path)
103 data_path = local_path
104 return load(data_path)
105
106 def dump_image(self, line):
107 os.makedirs(self.img_root, exist_ok=True)

Callers 1

load_dataMethod · 0.95

Calls 7

LMUDataRootFunction · 0.85
md5Function · 0.85
download_fileFunction · 0.85
file_sizeFunction · 0.85
LOCALIZEFunction · 0.85
loadFunction · 0.85
getMethod · 0.80

Tested by

no test coverage detected