MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / as_dataset

Method as_dataset

modelzoo/features/pmem/criteo.py:147–155  ·  view source on GitHub ↗
(self, shard=0, num_shards=1)

Source from the content-addressed store, hash-verified

145 return feats
146
147 def as_dataset(self, shard=0, num_shards=1):
148 works = tf.data.Dataset.from_tensor_slices(self._partition(shard, num_shards))
149 work_reader = tf.data.experimental.parallel_interleave(
150 self._make_dataset,
151 cycle_length=self.reader_threads,
152 block_length=self.reader_buffer,
153 sloppy=True)
154 ds = works.apply(work_reader)
155 return ds.map(self._next, self.parser_threads).prefetch(self.parser_buffer)

Callers 1

mainFunction · 0.95

Calls 5

_partitionMethod · 0.95
from_tensor_slicesMethod · 0.45
applyMethod · 0.45
prefetchMethod · 0.45
mapMethod · 0.45

Tested by

no test coverage detected