Create a server and continuously push examples from a generator. Args: example_generator: Generator that yields examples. port: Port for the gRPC server to listen on. max_queue_size: Number of example to queue. queue_timeout: Queue timeout argument. None to block forever, otherwis
(example_generator, port, max_queue_size, queue_timeout)
| 75 | |
| 76 | |
| 77 | def push_examples(example_generator, port, max_queue_size, queue_timeout): |
| 78 | """Create a server and continuously push examples from a generator. |
| 79 | |
| 80 | Args: |
| 81 | example_generator: Generator that yields examples. |
| 82 | port: Port for the gRPC server to listen on. |
| 83 | max_queue_size: Number of example to queue. |
| 84 | queue_timeout: Queue timeout argument. None to block forever, otherwise time |
| 85 | in seconds. |
| 86 | """ |
| 87 | servicer = PreprocessingServicer(max_queue_size=max_queue_size) |
| 88 | server = create_and_start_server(port, servicer) |
| 89 | for i, example in enumerate(example_generator): |
| 90 | servicer.PushExample(example, timeout=queue_timeout) |
| 91 | if i % 500 == 0: |
| 92 | tf.logging.info("Attempted to push %d examples.", i) |
| 93 | tf.logging.info("Queue size is %d.", servicer.QueueSize()) |
| 94 | server.stop(120.0) # The argument is grace period in seconds. |
| 95 | |
| 96 | |
| 97 | def _make_rpc_op(address, timeout_in_ms): |
nothing calls this directly
no test coverage detected
searching dependent graphs…