MCPcopy Create free account
hub / github.com/apache/arrow / test_write_dataset_partitioned

Function test_write_dataset_partitioned

python/pyarrow/tests/test_dataset.py:4262–4290  ·  view source on GitHub ↗
(tempdir)

Source from the content-addressed store, hash-verified

4260@pytest.mark.parquet
4261@pytest.mark.pandas
4262def test_write_dataset_partitioned(tempdir):
4263 directory = tempdir / "partitioned"
4264 _ = _create_parquet_dataset_partitioned(directory)
4265 partitioning = ds.partitioning(flavor="hive")
4266 dataset = ds.dataset(directory, partitioning=partitioning)
4267
4268 # hive partitioning
4269 target = tempdir / 'partitioned-hive-target'
4270 expected_paths = [
4271 target / "part=a", target / "part=a" / "part-0.arrow",
4272 target / "part=b", target / "part=b" / "part-0.arrow"
4273 ]
4274 partitioning_schema = ds.partitioning(
4275 pa.schema([("part", pa.string())]), flavor="hive")
4276 _check_dataset_roundtrip(
4277 dataset, str(target), expected_paths, 'f1', target,
4278 partitioning=partitioning_schema)
4279
4280 # directory partitioning
4281 target = tempdir / 'partitioned-dir-target'
4282 expected_paths = [
4283 target / "a", target / "a" / "part-0.arrow",
4284 target / "b", target / "b" / "part-0.arrow"
4285 ]
4286 partitioning_schema = ds.partitioning(
4287 pa.schema([("part", pa.string())]))
4288 _check_dataset_roundtrip(
4289 dataset, str(target), expected_paths, 'f1', target,
4290 partitioning=partitioning_schema)
4291
4292
4293def test_write_dataset_with_field_names(tempdir):

Callers

nothing calls this directly

Calls 5

_check_dataset_roundtripFunction · 0.85
partitioningMethod · 0.80
schemaMethod · 0.45
stringMethod · 0.45

Tested by

no test coverage detected