MCPcopy
hub / github.com/huggingface/datasets / test_write_batch

Method test_write_batch

tests/features/test_array_xd.py:223–233  ·  view source on GitHub ↗
(self, array_feature, shape_1, shape_2)

Source from the content-addressed store, hash-verified

221 del dataset
222
223 def test_write_batch(self, array_feature, shape_1, shape_2):
224 with tempfile.TemporaryDirectory() as tmp_dir:
225 my_features = self.get_features(array_feature, shape_1, shape_2)
226 dict_examples = self.get_dict_examples(shape_1, shape_2)
227 dict_examples = my_features.encode_batch(dict_examples)
228 with ArrowWriter(features=my_features, path=os.path.join(tmp_dir, "beta.arrow")) as writer:
229 writer.write_batch(dict_examples)
230 num_examples, num_bytes = writer.finalize()
231 dataset = datasets.Dataset.from_file(os.path.join(tmp_dir, "beta.arrow"))
232 self._check_getitem_output_type(dataset, shape_1, shape_2, dict_examples["matrix"][0])
233 del dataset
234
235 def test_from_dict(self, array_feature, shape_1, shape_2):
236 dict_examples = self.get_dict_examples(shape_1, shape_2)

Callers

nothing calls this directly

Calls 8

get_featuresMethod · 0.95
get_dict_examplesMethod · 0.95
ArrowWriterClass · 0.90
encode_batchMethod · 0.80
write_batchMethod · 0.80
finalizeMethod · 0.80
from_fileMethod · 0.45

Tested by

no test coverage detected