MCPcopy Create free account
hub / github.com/apache/arrow / test_union_dataset_filesystem_datasets

Function test_union_dataset_filesystem_datasets

python/pyarrow/tests/test_dataset.py:3266–3295  ·  view source on GitHub ↗
(multisourcefs)

Source from the content-addressed store, hash-verified

3264
3265
3266def test_union_dataset_filesystem_datasets(multisourcefs):
3267 # without partitioning
3268 dataset = ds.dataset([
3269 ds.dataset('/plain', filesystem=multisourcefs),
3270 ds.dataset('/schema', filesystem=multisourcefs),
3271 ds.dataset('/hive', filesystem=multisourcefs),
3272 ])
3273 expected_schema = pa.schema([
3274 ('date', pa.date32()),
3275 ('index', pa.int64()),
3276 ('value', pa.float64()),
3277 ('color', pa.string()),
3278 ])
3279 assert dataset.schema.equals(expected_schema)
3280
3281 # with hive partitioning for two hive sources
3282 dataset = ds.dataset([
3283 ds.dataset('/plain', filesystem=multisourcefs),
3284 ds.dataset('/schema', filesystem=multisourcefs),
3285 ds.dataset('/hive', filesystem=multisourcefs, partitioning='hive')
3286 ])
3287 expected_schema = pa.schema([
3288 ('date', pa.date32()),
3289 ('index', pa.int64()),
3290 ('value', pa.float64()),
3291 ('color', pa.string()),
3292 ('year', pa.int32()),
3293 ('month', pa.int32()),
3294 ])
3295 assert dataset.schema.equals(expected_schema)
3296
3297
3298@pytest.mark.parquet

Callers

nothing calls this directly

Calls 3

equalsMethod · 0.80
schemaMethod · 0.45
stringMethod · 0.45

Tested by

no test coverage detected