MCPcopy Create free account
hub / github.com/deepspeedai/DeepSpeed / parallel_for

Method parallel_for

csrc/adam/cpu_adam_impl.cpp:406–419  ·  view source on GitHub ↗

Split [0, total) into one chunk per thread. Chunk boundaries are rounded up to a multiple of `align` so each slice's AVX/scalar split lines up with the whole-tensor kernel's split -- otherwise an element could be computed by AVX (FMA) in one layout and the scalar tail (mul+add) in another, which differ in the last bit.

Source from the content-addressed store, hash-verified

404 // kernel's split -- otherwise an element could be computed by AVX (FMA) in one layout
405 // and the scalar tail (mul+add) in another, which differ in the last bit.
406 void parallel_for(size_t total, size_t align, std::function<void(size_t, size_t)> fn)
407 {
408 {
409 std::unique_lock<std::mutex> lk(m_);
410 fn_ = std::move(fn);
411 total_ = total;
412 align_ = std::max<size_t>(1, align);
413 done_count_ = 0;
414 ++gen_;
415 }
416 cv_start_.notify_all();
417 std::unique_lock<std::mutex> lk(m_);
418 cv_done_.wait(lk, [this] { return done_count_ == n_; });
419 }
420
421private:
422 void worker(size_t tid, int core)

Callers 4

launch_param_updateFunction · 0.80
launch_param_update_halfFunction · 0.80
multi_tensor_applyFunction · 0.80
run_stepMethod · 0.80

Calls 2

moveFunction · 0.85
waitMethod · 0.45

Tested by

no test coverage detected