MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / Shard

Function Shard

tensorflow/core/util/work_sharder.cc:32–51  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

30int GetPerThreadMaxParallelism() { return per_thread_max_parallism; }
31
32void Shard(int max_parallelism, thread::ThreadPool* workers, int64 total,
33 int64 cost_per_unit, std::function<void(int64, int64)> work) {
34 CHECK_GE(total, 0);
35 if (total == 0) {
36 return;
37 }
38 max_parallelism = std::min(max_parallelism, GetPerThreadMaxParallelism());
39 if (max_parallelism <= 1) {
40 // Just inline the whole work since we only have 1 thread (core).
41 work(0, total);
42 return;
43 }
44 if (max_parallelism >= workers->NumThreads()) {
45 workers->ParallelFor(total, cost_per_unit, work);
46 return;
47 }
48 Sharder::Do(total, cost_per_unit, work,
49 [&workers](Sharder::Closure c) { workers->Schedule(c); },
50 max_parallelism);
51}
52
53// DEPRECATED: Prefer threadpool->TransformRangeConcurrently, which allows you
54// to directly specify the shard size.

Callers 15

RunShardingFunction · 0.70
TESTFunction · 0.70
BM_ShardingFunction · 0.70
do_periodic_resample_opFunction · 0.50
DoComputeMethod · 0.50
operator()Method · 0.50
ComputeMethod · 0.50
ComputeMethod · 0.50
ComputeMethod · 0.50
ComputeMethod · 0.50
operator()Method · 0.50
ComputeMethod · 0.50

Calls 5

ParallelForMethod · 0.80
minFunction · 0.50
NumThreadsMethod · 0.45
ScheduleMethod · 0.45

Tested by 3

RunShardingFunction · 0.56
TESTFunction · 0.56
BM_ShardingFunction · 0.56