MCPcopy
hub / github.com/dask/dask / test_read_bytes_delimited

Function test_read_bytes_delimited

dask/bytes/tests/test_local.py:166–189  ·  view source on GitHub ↗
()

Source from the content-addressed store, hash-verified

164
165
166def test_read_bytes_delimited():
167 with filetexts(files, mode="b"):
168 for bs in [5, 15, 45, "1.5 kB"]:
169 _, values = read_bytes(".test.accounts*", blocksize=bs, delimiter=b"\n")
170 _, values2 = read_bytes(".test.accounts*", blocksize=bs, delimiter=b"foo")
171 assert [a.key for a in concat(values)] != [b.key for b in concat(values2)]
172
173 results = compute(*concat(values))
174 res = [r for r in results if r]
175 assert all(r.endswith(b"\n") for r in res)
176 ourlines = b"".join(res).split(b"\n")
177 testlines = b"".join(files[k] for k in sorted(files)).split(b"\n")
178 assert ourlines == testlines
179
180 # delimiter not at the end
181 d = b"}"
182 _, values = read_bytes(".test.accounts*", blocksize=bs, delimiter=d)
183 results = compute(*concat(values))
184 res = [r for r in results if r]
185 # All should end in } except EOF
186 assert sum(r.endswith(b"}") for r in res) == len(res) - 2
187 ours = b"".join(res)
188 test = b"".join(files[v] for v in sorted(files))
189 assert ours == test
190
191
192fmt_bs = [(fmt, None) for fmt in compr] + [(fmt, 10) for fmt in compr]

Callers

nothing calls this directly

Calls 8

filetextsFunction · 0.90
read_bytesFunction · 0.90
computeFunction · 0.90
allFunction · 0.85
splitMethod · 0.80
concatFunction · 0.50
sumFunction · 0.50
joinMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…