MCPcopy Create free account
hub / github.com/apache/arrow / read_table

Function read_table

python/pyarrow/parquet/core.py:1903–1986  ·  view source on GitHub ↗
(source, *, columns=None, use_threads=True,
               schema=None, use_pandas_metadata=False, read_dictionary=None,
               binary_type=None, list_type=None, memory_map=False, buffer_size=0,
               partitioning="hive", filesystem=None, filters=None,
               ignore_prefixes=None, pre_buffer=True,
               coerce_int96_timestamp_unit=None,
               decryption_properties=None, thrift_string_size_limit=None,
               thrift_container_size_limit=None, schema_depth_limit=None,
               page_checksum_verification=False,
               arrow_extensions_enabled=True)

Source from the content-addressed store, hash-verified

1901
1902
1903def read_table(source, *, columns=None, use_threads=True,
1904 schema=None, use_pandas_metadata=False, read_dictionary=None,
1905 binary_type=None, list_type=None, memory_map=False, buffer_size=0,
1906 partitioning="hive", filesystem=None, filters=None,
1907 ignore_prefixes=None, pre_buffer=True,
1908 coerce_int96_timestamp_unit=None,
1909 decryption_properties=None, thrift_string_size_limit=None,
1910 thrift_container_size_limit=None, schema_depth_limit=None,
1911 page_checksum_verification=False,
1912 arrow_extensions_enabled=True):
1913
1914 try:
1915 dataset = ParquetDataset(
1916 source,
1917 schema=schema,
1918 filesystem=filesystem,
1919 partitioning=partitioning,
1920 memory_map=memory_map,
1921 read_dictionary=read_dictionary,
1922 binary_type=binary_type,
1923 list_type=list_type,
1924 buffer_size=buffer_size,
1925 filters=filters,
1926 ignore_prefixes=ignore_prefixes,
1927 pre_buffer=pre_buffer,
1928 coerce_int96_timestamp_unit=coerce_int96_timestamp_unit,
1929 decryption_properties=decryption_properties,
1930 thrift_string_size_limit=thrift_string_size_limit,
1931 thrift_container_size_limit=thrift_container_size_limit,
1932 schema_depth_limit=schema_depth_limit,
1933 page_checksum_verification=page_checksum_verification,
1934 arrow_extensions_enabled=arrow_extensions_enabled,
1935 )
1936 except ImportError:
1937 # fall back on ParquetFile for simple cases when pyarrow.dataset
1938 # module is not available
1939 if filters is not None:
1940 raise ValueError(
1941 "the 'filters' keyword is not supported when the "
1942 "pyarrow.dataset module is not available"
1943 )
1944 if partitioning != "hive":
1945 raise ValueError(
1946 "the 'partitioning' keyword is not supported when the "
1947 "pyarrow.dataset module is not available"
1948 )
1949 if schema is not None:
1950 raise ValueError(
1951 "the 'schema' argument is not supported when the "
1952 "pyarrow.dataset module is not available"
1953 )
1954 if isinstance(source, list):
1955 raise ValueError(
1956 "the 'source' argument cannot be a list of files "
1957 "when the pyarrow.dataset module is not available"
1958 )
1959
1960 filesystem, path = _resolve_filesystem_and_path(source, filesystem)

Callers 1

read_pandasFunction · 0.70

Calls 6

readMethod · 0.95
ParquetDatasetClass · 0.85
ParquetFileClass · 0.85
get_file_infoMethod · 0.45
open_input_fileMethod · 0.45

Tested by

no test coverage detected