MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / TryDequeueMany

Method TryDequeueMany

tensorflow/core/kernels/priority_queue.cc:238–386  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

236}
237
238void PriorityQueue::TryDequeueMany(int num_elements, OpKernelContext* ctx,
239 bool allow_small_batch,
240 CallbackWithTuple callback) {
241 if (!specified_shapes()) {
242 ctx->SetStatus(
243 errors::InvalidArgument("PriorityQueue's DequeueMany requires the "
244 "components to have specified shapes."));
245 callback(Tuple());
246 return;
247 }
248 if (num_elements == 0) {
249 Tuple tuple;
250 tuple.reserve(num_components());
251 for (int i = 0; i < num_components(); ++i) {
252 // TODO(josh11b,misard): Switch to allocate_output(). Problem is
253 // this breaks the abstraction boundary since we don't *really*
254 // know if and how the Tensors in the tuple we pass to callback
255 // correspond to the outputs of *ctx. For example, the
256 // ReaderRead Op uses TryDequeue() to get a filename out of a
257 // queue that is used internally by the reader and is not
258 // associated with any output of the ReaderRead.
259 // mrry@ adds:
260 // Maybe we need to pass a std::function<Tensor*(...)> (or
261 // better signature) that calls the appropriate allocator
262 // function in addition to ctx? (Or support a shim Allocator
263 // that has an internal OpKernelContext*, and dispatches to the
264 // appropriate method?)
265 // misard@ adds:
266 // I don't see that a std::function would help. The problem is
267 // that at this point (allocation time) the system doesn't know
268 // what is going to happen to the element read out of the
269 // queue. As long as we keep the generality that TensorFlow Ops
270 // do their own dynamic allocation in arbitrary C++ code, we
271 // need to preserve robustness to allocating output Tensors with
272 // the 'wrong' attributes, and fixing up with a copy. The only
273 // improvement I can see here in the future would be to support
274 // an optimized case where the queue 'knows' what attributes to
275 // use, and plumbs them through here.
276 Tensor element;
277 Status status = ctx->allocate_temp(component_dtypes_[i],
278 ManyOutShape(i, 0), &element);
279 if (!status.ok()) {
280 ctx->SetStatus(status);
281 callback(Tuple());
282 return;
283 }
284 tuple.emplace_back(element);
285 }
286 callback(tuple);
287 return;
288 }
289
290 CancellationManager* cm = ctx->cancellation_manager();
291 CancellationToken token = cm->get_cancellation_token();
292 bool already_cancelled;
293 {
294 mutex_lock l(mu_);
295 already_cancelled = !cm->RegisterCallback(

Callers

nothing calls this directly

Calls 15

specified_shapesFunction · 0.85
InvalidArgumentFunction · 0.85
callbackFunction · 0.85
num_componentsFunction · 0.85
ManyOutShapeFunction · 0.85
CopyElementToSliceFunction · 0.85
CancelledFunction · 0.85
TupleFunction · 0.50
EXCLUSIVE_LOCKS_REQUIREDFunction · 0.50
SetStatusMethod · 0.45
reserveMethod · 0.45

Tested by

no test coverage detected