MCPcopy
hub / github.com/InternLM/lmdeploy / send_response_loop

Method send_response_loop

lmdeploy/pytorch/engine/engine_loop.py:244–255  ·  view source on GitHub ↗

Send response to client.

(self)

Source from the content-addressed store, hash-verified

242 self._send_resp(out)
243
244 async def send_response_loop(self):
245 """Send response to client."""
246 que = self.resp_queue
247 while not self.stop_event.is_set():
248 num_outs = que.qsize()
249 if num_outs > 0:
250 resps = []
251 for _ in range(num_outs):
252 resps += que.get_nowait().values()
253 else:
254 resps = (await que.get()).values()
255 self._send_resps(resps)
256
257 @record_function('make_infer_outputs')
258 def _make_infer_outputs(

Callers 1

startMethod · 0.95

Calls 3

_send_respsMethod · 0.95
valuesMethod · 0.80
getMethod · 0.45

Tested by

no test coverage detected