Graph dataset. Parameters: name (str): name of dataset urls (dict, optional): url(s) for each split, can be either str or list of str members (dict, optional): zip member(s) for each split, leave empty for default Datasets contain severa
| 60 | |
| 61 | |
| 62 | class Dataset(object): |
| 63 | """ |
| 64 | Graph dataset. |
| 65 | |
| 66 | Parameters: |
| 67 | name (str): name of dataset |
| 68 | urls (dict, optional): url(s) for each split, |
| 69 | can be either str or list of str |
| 70 | members (dict, optional): zip member(s) for each split, |
| 71 | leave empty for default |
| 72 | |
| 73 | Datasets contain several splits, such as train, valid and test. |
| 74 | For each split, there are one or more URLs, specifying the file to download. |
| 75 | You may also specify the zip member to extract. |
| 76 | When a split is accessed, it will be automatically downloaded and decompressed |
| 77 | if it is not present. |
| 78 | |
| 79 | You can assign a preprocess for each split, by defining a function with name [split]_preprocess:: |
| 80 | |
| 81 | class MyDataset(Dataset): |
| 82 | def __init__(self): |
| 83 | super(MyDataset, self).__init__( |
| 84 | "my_dataset", |
| 85 | train="url/to/train/split", |
| 86 | test="url/to/test/split" |
| 87 | ) |
| 88 | |
| 89 | def train_preprocess(self, input_file, output_file): |
| 90 | with open(input_file, "r") as fin, open(output_file, "w") as fout: |
| 91 | fout.write(fin.read()) |
| 92 | |
| 93 | f = open(MyDataset().train) |
| 94 | |
| 95 | If the preprocess returns a non-trivial value, then it is assigned to the split, |
| 96 | otherwise the file name is assigned. |
| 97 | By convention, only splits ending with ``_data`` have non-trivial return value. |
| 98 | |
| 99 | See also: |
| 100 | Pre-defined preprocess functions |
| 101 | :func:`csv2txt`, |
| 102 | :func:`top_k_label`, |
| 103 | :func:`induced_graph`, |
| 104 | :func:`edge_split`, |
| 105 | :func:`link_prediction_split`, |
| 106 | :func:`image_feature_data` |
| 107 | """ |
| 108 | def __init__(self, name, urls=None, members=None): |
| 109 | self.name = name |
| 110 | self.urls = urls or {} |
| 111 | self.members = members or {} |
| 112 | for key in self.urls: |
| 113 | if isinstance(self.urls[key], str): |
| 114 | self.urls[key] = [self.urls[key]] |
| 115 | if key not in self.members: |
| 116 | self.members[key] = [None] * len(self.urls[key]) |
| 117 | elif isinstance(self.members[key], str): |
| 118 | self.members[key] = [self.members[key]] |
| 119 | if len(self.urls[key]) != len(self.members[key]): |
nothing calls this directly
no outgoing calls
no test coverage detected