If _build_dag_each_worker is True, Starting _worker_num processes and running one GRPC server in each process. Otherwise, Staring on

save_to_file

seek

查询

when opening tensorrt(configure in config.yml) and each time the input shape for inferring is different, using this method for config

Set the name of the Op as the key of the matching engine. Notes that this key is only used by infer_op (only the infer_op needs to find the correspond

Set input ops.Each op have many input ops, but only one input channel. Args: ops: op list Returns:

Set jump to ops, then, this op can send channeldata to output channel. Args: ops: op list to be jumpped Returns

根据key值及分片数判断去哪一个分片上查询

Start multiple processes and start one server in each process

Each OP creates a process to run the main loop, initializes the CUDA environment in each individual process. Args:

Each OP creates a thread to run the main loop, initializes the CUDA environment in the main thread. Args: None

Stopping and cleanning all channels. Args: None Returns: None

Return a mutable char* pointing to a string's internal buffer, which may not be null-terminated. Writing through this pointer will modify the string.

This method gives an example of synchronous prediction whose input type is string.

For each feedvar, batch should be 1 or batch_size. if feedvar-1: batch_size = 1 (always not batch). feedvar-2: batch_size = n, batch = n. this functi

tell