MCPcopy Create free account
hub / github.com/dask/dask / tree_width

Function tree_width

dask/dataframe/partitionquantiles.py:162–177  ·  view source on GitHub ↗

Generate tree width suitable for ``merge_sorted`` given N inputs The larger N is, the more tasks are reduced in a single task. In theory, this is designed so all tasks are of comparable effort.

(N, to_binary=False)

Source from the content-addressed store, hash-verified

160
161
162def tree_width(N, to_binary=False):
163 """Generate tree width suitable for ``merge_sorted`` given N inputs
164
165 The larger N is, the more tasks are reduced in a single task.
166
167 In theory, this is designed so all tasks are of comparable effort.
168 """
169 if N < 32:
170 group_size = 2
171 else:
172 group_size = int(math.log(N))
173 num_groups = N // group_size
174 if to_binary or num_groups < 16:
175 return 2 ** int(math.log2(N / group_size))
176 else:
177 return num_groups
178
179
180def tree_groups(N, num_groups):

Callers 1

create_merge_treeFunction · 0.85

Calls

no outgoing calls

Tested by

no test coverage detected