MCPcopy Create free account
hub / github.com/apache/arrow / _get_compare_pair

Function _get_compare_pair

python/pyarrow/tests/test_dataset.py:4645–4649  ·  view source on GitHub ↗
(data_source, record_batch, file_format, col_id)

Source from the content-addressed store, hash-verified

4643 # partitioning column contains duplicate records.
4644 # Returns: (number_of_files_generated, number_of_partitions)
4645 def _get_compare_pair(data_source, record_batch, file_format, col_id):
4646 num_of_files_generated = _get_num_of_files_generated(
4647 base_directory=data_source, file_format=file_format)
4648 number_of_partitions = len(pa.compute.unique(record_batch[col_id]))
4649 return num_of_files_generated, number_of_partitions
4650
4651 # CASE 1: when max_open_files=default & max_open_files >= num_of_partitions
4652 # In case of a writing to disk via partitioning based on a

Callers 1

Calls 3

lenFunction · 0.85
uniqueMethod · 0.45

Tested by

no test coverage detected