Function _intersect_1d

dask/array/rechunk.py:60–156

Internal utility to intersect chunks for 1d after preprocessing. >>> new = cumdims_label(((2, 3), (2, 2, 1)), 'n') >>> old = cumdims_label(((2, 2, 1), (5,)), 'o') >>> _intersect_1d(_breakpoints(old[0], new[0])) # doctest: +NORMALIZE_WHITESPACE [[(0, slice(0, 2, None))],

(breaks)

def _intersect_1d(breaks):
    """
    Internal utility to intersect chunks for 1d after preprocessing.

    >>> new = cumdims_label(((2, 3), (2, 2, 1)), 'n')
    >>> old = cumdims_label(((2, 2, 1), (5,)), 'o')

    >>> _intersect_1d(_breakpoints(old[0], new[0]))  # doctest: +NORMALIZE_WHITESPACE
    [[(0, slice(0, 2, None))],
     [(1, slice(0, 2, None)), (2, slice(0, 1, None))]]
    >>> _intersect_1d(_breakpoints(old[1], new[1]))  # doctest: +NORMALIZE_WHITESPACE
    [[(0, slice(0, 2, None))],
     [(0, slice(2, 4, None))],
     [(0, slice(4, 5, None))]]

    Parameters
    ----------

    breaks: list of tuples
        Each tuple is ('o', 8) or ('n', 8)
        These are pairs of 'o' old or new 'n'
        indicator with a corresponding cumulative sum,
        or breakpoint (a position along the chunking axis).
        The list of pairs is already ordered by breakpoint.
        Note that an 'o' pair always occurs BEFORE
        an 'n' pair if both share the same breakpoint.
    Uses 'o' and 'n' to make new tuples of slices for
    the new block crosswalk to old blocks.
    """
    # EXPLANATION:
    # We know each new chunk is obtained from the old chunks, but
    # from which ones and how? This function provides the answer.
    # On return, each new chunk is represented as a list of slices
    # of the old chunks. Therefore,  paired with each slice is the
    # index of the old chunk to which that slice refers.
    # NOTE: if any nonzero-size new chunks extend beyond the total
    #    span of the old chunks, then those new chunks are assumed
    #    to be obtained from an imaginary old chunk that extends
    #    from the end of that total span to infinity. The chunk-
    #    index of this imaginary chunk follows in consecutive order
    #    from the chunk-indices of the actual old chunks.

    # First, let us determine the index of the last old_chunk:
    o_pairs = [pair for pair in breaks if pair[0] == "o"]
    last_old_chunk_idx = len(o_pairs) - 2
    last_o_br = o_pairs[-1][1]  # end of range spanning all old chunks

    start = 0  # start of a slice of an old chunk
    last_end = 0
    old_idx = 0  # index of old chunk
    last_o_end = 0
    ret = []  # will hold the list of new chunks
    ret_next = []  # will hold the list of slices comprising one new chunk
    for idx in range(1, len(breaks)):  # Note start from the 2nd pair
        # the interval between any two consecutive breakpoints is a potential
        # new chunk:
        label, br = breaks[idx]
        last_label, last_br = breaks[idx - 1]