MCPcopy Index your code
hub / github.com/diffgram/diffgram / __fetch_folder

Method __fetch_folder

shared/connection/s3_connector.py:173–197  ·  view source on GitHub ↗
(self, opts)

Source from the content-addressed store, hash-verified

171
172 @with_connection
173 def __fetch_folder(self, opts):
174 paths = opts['path']
175 if type(paths) != list:
176 paths = [paths]
177 for current_path in paths:
178 kwargs = {'Bucket': opts['bucket_name'], 'Prefix': current_path}
179 while True:
180 resp = self.connection_client.list_objects_v2(**kwargs)
181 if resp.get('Contents'):
182 for obj in resp['Contents']:
183 if not obj['Key'].endswith('/'):
184 opts_fetch_object = {}
185 opts_fetch_object.update(opts)
186 new_opts = {
187 'path': obj['Key'],
188 'directory_id': opts.get('directory_id'),
189 'batch_id': opts.get('batch_id'),
190 'bucket_name': opts.get('bucket_name'),
191 }
192 opts_fetch_object.update(new_opts)
193 self.__fetch_object(opts_fetch_object)
194 try:
195 kwargs['ContinuationToken'] = resp['NextContinuationToken']
196 except KeyError:
197 break
198
199 @with_connection
200 @with_s3_exception_handler

Callers

nothing calls this directly

Calls 3

__fetch_objectMethod · 0.95
getMethod · 0.45
updateMethod · 0.45

Tested by

no test coverage detected