MCPcopy Create free account
hub / github.com/apache/arrow / test_stress_block_sizes

Method test_stress_block_sizes

python/pyarrow/tests/test_csv.py:1897–1913  ·  view source on GitHub ↗
(self)

Source from the content-addressed store, hash-verified

1895 assert table.to_pydict() == expected.to_pydict()
1896
1897 def test_stress_block_sizes(self):
1898 # Test a number of small block sizes to stress block stitching
1899 csv_base, expected = make_random_csv(num_cols=2, num_rows=500)
1900 block_sizes = [19, 21, 23, 26, 37, 111]
1901 csvs = [csv_base, csv_base.rstrip(b'\r\n')]
1902 for csv in csvs:
1903 for block_size in block_sizes:
1904 # Need at least two lines for type inference
1905 assert csv[:block_size].count(b'\n') >= 2
1906 read_options = ReadOptions(block_size=block_size)
1907 reader = self.open_bytes(
1908 csv, read_options=read_options)
1909 table = reader.read_all()
1910 assert table.schema == expected.schema
1911 if not table.equals(expected):
1912 # Better error output
1913 assert table.to_pydict() == expected.to_pydict()
1914
1915 def test_batch_lifetime(self):
1916 gc.collect()

Callers

nothing calls this directly

Calls 5

open_bytesMethod · 0.95
ReadOptionsClass · 0.90
make_random_csvFunction · 0.85
equalsMethod · 0.80
countMethod · 0.45

Tested by

no test coverage detected