MCPcopy Create free account
hub / github.com/apache/arrow / test_stress_block_sizes

Method test_stress_block_sizes

python/pyarrow/tests/test_csv.py:1896–1912  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

1894 assert table.to_pydict() == expected.to_pydict()
1895
1896 def test_stress_block_sizes(self):
1897 # Test a number of small block sizes to stress block stitching
1898 csv_base, expected = make_random_csv(num_cols=2, num_rows=500)
1899 block_sizes = [19, 21, 23, 26, 37, 111]
1900 csvs = [csv_base, csv_base.rstrip(b'\r\n')]
1901 for csv in csvs:
1902 for block_size in block_sizes:
1903 # Need at least two lines for type inference
1904 assert csv[:block_size].count(b'\n') >= 2
1905 read_options = ReadOptions(block_size=block_size)
1906 reader = self.open_bytes(
1907 csv, read_options=read_options)
1908 table = reader.read_all()
1909 assert table.schema == expected.schema
1910 if not table.equals(expected):
1911 # Better error output
1912 assert table.to_pydict() == expected.to_pydict()
1913
1914 def test_batch_lifetime(self):
1915 gc.collect()

Callers

nothing calls this directly

Calls 5

open_bytesMethod · 0.95
ReadOptionsClass · 0.90
make_random_csvFunction · 0.85
equalsMethod · 0.80
countMethod · 0.45

Tested by

no test coverage detected