MCPcopy Create free account
hub / github.com/apache/arrow / test_do_put_independent_read_write

Function test_do_put_independent_read_write

python/pyarrow/tests/test_flight.py:2100–2136  ·  view source on GitHub ↗

Ensure that separate threads can read/write on a DoPut.

()

Source from the content-addressed store, hash-verified

2098
2099
2100def test_do_put_independent_read_write():
2101 """Ensure that separate threads can read/write on a DoPut."""
2102 # ARROW-6063: previously this would cause gRPC to abort when the
2103 # writer was closed (due to simultaneous reads), or would hang
2104 # forever.
2105 data = [
2106 pa.array([-10, -5, 0, 5, 10])
2107 ]
2108 table = pa.Table.from_arrays(data, names=['a'])
2109
2110 with MetadataFlightServer() as server, \
2111 FlightClient(('localhost', server.port)) as client:
2112 writer, metadata_reader = client.do_put(
2113 flight.FlightDescriptor.for_path(''),
2114 table.schema)
2115
2116 count = [0]
2117
2118 def _reader_thread():
2119 while metadata_reader.read() is not None:
2120 count[0] += 1
2121
2122 thread = threading.Thread(target=_reader_thread)
2123 thread.start()
2124
2125 batches = table.to_batches(max_chunksize=1)
2126 with writer:
2127 for idx, batch in enumerate(batches):
2128 metadata = struct.pack('<i', idx)
2129 writer.write_with_metadata(batch, metadata)
2130 # Causes the server to stop writing and end the call
2131 writer.done_writing()
2132 # Thus reader thread will break out of loop
2133 thread.join()
2134 # writer.close() won't segfault since reader thread has
2135 # stopped
2136 assert count[0] == len(batches)
2137
2138
2139def test_server_middleware_same_thread():

Callers

nothing calls this directly

Calls 9

FlightClientClass · 0.90
lenFunction · 0.85
to_batchesMethod · 0.80
arrayMethod · 0.45
do_putMethod · 0.45
startMethod · 0.45
packMethod · 0.45
joinMethod · 0.45

Tested by

no test coverage detected