MCPcopy Create free account
hub / github.com/dask/dask / store

Function store

dask/array/core.py:1086–1237  ·  view source on GitHub ↗

Store dask arrays in array-like objects, overwrite data in target This stores dask arrays into object that supports numpy-style setitem indexing. It stores values chunk by chunk so that it does not have to fill up memory. For best performance you can align the block size of the st

(
    sources: Array | Collection[Array],
    targets: ArrayLike | Delayed | Collection[ArrayLike | Delayed],
    lock: bool | Lock = True,
    regions: tuple[slice, ...] | Collection[tuple[slice, ...]] | None = None,
    compute: bool = True,
    return_stored: bool = False,
    load_stored: bool | None = None,
    **kwargs,
)

Source from the content-addressed store, hash-verified

1084
1085
1086def store(
1087 sources: Array | Collection[Array],
1088 targets: ArrayLike | Delayed | Collection[ArrayLike | Delayed],
1089 lock: bool | Lock = True,
1090 regions: tuple[slice, ...] | Collection[tuple[slice, ...]] | None = None,
1091 compute: bool = True,
1092 return_stored: bool = False,
1093 load_stored: bool | None = None,
1094 **kwargs,
1095):
1096 """Store dask arrays in array-like objects, overwrite data in target
1097
1098 This stores dask arrays into object that supports numpy-style setitem
1099 indexing. It stores values chunk by chunk so that it does not have to
1100 fill up memory. For best performance you can align the block size of
1101 the storage target with the block size of your array.
1102
1103 If your data fits in memory then you may prefer calling
1104 ``np.array(myarray)`` instead.
1105
1106 Parameters
1107 ----------
1108
1109 sources: Array or collection of Arrays
1110 targets: array-like or Delayed or collection of array-likes and/or Delayeds
1111 These should support setitem syntax ``target[10:20] = ...``.
1112 If sources is a single item, targets must be a single item; if sources is a
1113 collection of arrays, targets must be a matching collection.
1114 lock: boolean or threading.Lock, optional
1115 Whether or not to lock the data stores while storing.
1116 Pass True (lock each file individually), False (don't lock) or a
1117 particular :class:`threading.Lock` object to be shared among all writes.
1118 regions: tuple of slices or collection of tuples of slices, optional
1119 Each ``region`` tuple in ``regions`` should be such that
1120 ``target[region].shape = source.shape``
1121 for the corresponding source and target in sources and targets,
1122 respectively. If this is a tuple, the contents will be assumed to be
1123 slices, so do not provide a tuple of tuples.
1124 compute: boolean, optional
1125 If true compute immediately; return :class:`dask.delayed.Delayed` otherwise.
1126 return_stored: boolean, optional
1127 Optionally return the stored result (default False).
1128 load_stored: boolean, optional
1129 Optionally return the stored result, loaded in to memory (default None).
1130 If None, ``load_stored`` is True if ``return_stored`` is True and
1131 ``compute`` is False. *This is an advanced option.*
1132 When False, store will return the appropriate ``target`` for each chunk that is stored.
1133 Directly computing this result is not what you want.
1134 Instead, you can use the returned ``target`` to execute followup operations to the store.
1135 kwargs:
1136 Parameters passed to compute/persist (only used if compute=True)
1137
1138 Returns
1139 -------
1140
1141 If return_stored=True
1142 tuple of Arrays
1143 If return_stored=False and compute=True

Callers 9

test_store_kwargsFunction · 0.90
test_storeFunction · 0.90
test_store_regionsFunction · 0.90
test_store_compute_falseFunction · 0.90
test_store_locksFunction · 0.90
storeMethod · 0.85
to_hdf5Function · 0.85

Calls 7

anyFunction · 0.90
get_scheduler_lockFunction · 0.90
ArraySliceDepClass · 0.90
persistFunction · 0.90
getMethod · 0.45
map_blocksMethod · 0.45
computeMethod · 0.45

Tested by 7

test_store_kwargsFunction · 0.72
test_storeFunction · 0.72
test_store_regionsFunction · 0.72
test_store_compute_falseFunction · 0.72
test_store_locksFunction · 0.72