MCPcopy Index your code
hub / github.com/alibaba/bigcomputing / loadDataset

Function loadDataset

DIEN/data_utils.py:633–655  ·  view source on GitHub ↗
(
    raw_path 
)

Source from the content-addressed store, hash-verified

631
632
633def loadDataset(
634 raw_path
635):
636 # dataset
637 #output_filename = "dienDataset_processed"
638
639 # read in the dataset
640 #pos = raw_path.rfind('/')
641 #data_path = raw_path[:pos]
642 #data_file = raw_path[pos:]
643 #npzfile = "." + data_file + "_processed.npz"
644
645 data_path = raw_path
646 max_length = 1000
647 # If already processed, just load it
648 file = preprocessDataset(data_path, max_length)
649
650 #train_data_file = [ 'sample_00','sample_01', 'sample_02', 'sample_03']
651 #train_data_file = [ 'sample_03']
652 #file.process2( train_data_file, 'train_sample')
653
654 test_data_file = ['test_sample_00', 'test_sample_01', 'test_sample_02', 'test_sample_03']
655 file.process2( test_data_file, 'test_sample')
656
657
658if __name__ == "__main__":

Callers 1

data_utils.pyFile · 0.85

Calls 2

preprocessDatasetClass · 0.85
process2Method · 0.80

Tested by

no test coverage detected