MCPcopy Create free account
hub / github.com/apache/arrow / test_csv_format_options

Function test_csv_format_options

python/pyarrow/tests/test_dataset.py:3533–3551  ·  view source on GitHub ↗
(tempdir, dataset_reader)

Source from the content-addressed store, hash-verified

3531
3532
3533def test_csv_format_options(tempdir, dataset_reader):
3534 path = str(tempdir / 'test.csv')
3535 with open(path, 'w') as sink:
3536 sink.write('skipped\ncol0\nfoo\nbar\n')
3537 dataset = ds.dataset(path, format='csv')
3538 result = dataset_reader.to_table(dataset)
3539 assert result.equals(
3540 pa.table({'skipped': pa.array(['col0', 'foo', 'bar'])}))
3541
3542 dataset = ds.dataset(path, format=ds.CsvFileFormat(
3543 read_options=pa.csv.ReadOptions(skip_rows=1)))
3544 result = dataset_reader.to_table(dataset)
3545 assert result.equals(pa.table({'col0': pa.array(['foo', 'bar'])}))
3546
3547 dataset = ds.dataset(path, format=ds.CsvFileFormat(
3548 read_options=pa.csv.ReadOptions(column_names=['foo'])))
3549 result = dataset_reader.to_table(dataset)
3550 assert result.equals(
3551 pa.table({'foo': pa.array(['skipped', 'col0', 'foo', 'bar'])}))
3552
3553
3554def test_csv_format_options_generate_columns(tempdir, dataset_reader):

Callers

nothing calls this directly

Calls 5

equalsMethod · 0.80
CsvFileFormatMethod · 0.80
writeMethod · 0.45
to_tableMethod · 0.45
arrayMethod · 0.45

Tested by

no test coverage detected