MCPcopy Create free account
hub / github.com/intel/neural-compressor / MyDataLoader

Class MyDataLoader

test/tensorflow/quantization/test_smooth_quant.py:53–73  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

51
52
53class MyDataLoader:
54 def __init__(self, dataset, batch_size=1):
55 self.dataset = dataset
56 self.batch_size = batch_size
57 self.length = math.ceil(len(dataset) / self.batch_size)
58
59 def __iter__(self):
60 images_list = []
61 labels_list = []
62 for _, (images, labels) in enumerate(self.dataset):
63 images = np.expand_dims(images, axis=0)
64 labels = np.expand_dims(labels, axis=0)
65 images_list.append(images[0])
66 labels_list.append(labels[0])
67 if self.batch_size == len(images_list):
68 yield (images_list, labels_list)
69 images_list = []
70 labels_list = []
71
72 def __len__(self):
73 return self.length
74
75
76class TestSmoothQuantTF3xNewApi(unittest.TestCase):

Callers 6

test_convMethod · 0.70
test_matmulMethod · 0.70
test_conv_matmulMethod · 0.70

Calls

no outgoing calls

Tested by 6

test_convMethod · 0.56
test_matmulMethod · 0.56
test_conv_matmulMethod · 0.56

Used in the wild real call sites across dependent graphs

searching dependent graphs…