MCPcopy Create free account
hub / github.com/apache/arrow / test_make_csv_fragment_from_buffer

Function test_make_csv_fragment_from_buffer

python/pyarrow/tests/test_dataset.py:1296–1318  ·  view source on GitHub ↗
(dataset_reader, pickle_module)

Source from the content-addressed store, hash-verified

1294
1295
1296def test_make_csv_fragment_from_buffer(dataset_reader, pickle_module):
1297 content = textwrap.dedent("""
1298 alpha,num,animal
1299 a,12,dog
1300 b,11,cat
1301 c,10,rabbit
1302 """)
1303 buffer = pa.py_buffer(content.encode('utf-8'))
1304
1305 csv_format = ds.CsvFileFormat()
1306 fragment = csv_format.make_fragment(buffer)
1307
1308 # When buffer, fragment open returns a BufferReader, not NativeFile
1309 assert isinstance(fragment.open(), pa.BufferReader)
1310
1311 expected = pa.table([['a', 'b', 'c'],
1312 [12, 11, 10],
1313 ['dog', 'cat', 'rabbit']],
1314 names=['alpha', 'num', 'animal'])
1315 assert dataset_reader.to_table(fragment).equals(expected)
1316
1317 pickled = pickle_module.loads(pickle_module.dumps(fragment))
1318 assert dataset_reader.to_table(pickled).equals(fragment.to_table())
1319
1320
1321def test_make_json_fragment_from_buffer(dataset_reader, pickle_module):

Callers

nothing calls this directly

Calls 7

CsvFileFormatMethod · 0.80
equalsMethod · 0.80
loadsMethod · 0.80
dumpsMethod · 0.80
encodeMethod · 0.45
openMethod · 0.45
to_tableMethod · 0.45

Tested by

no test coverage detected