Wikitext-2 dataset. This is a PyTorch dataset for the pre-processed Wikitext-2 dataset, containing about 2M words. This dataset will produce vectors of tokens of fixed size max_length, possibly zero-padded. Parameters ---------- root (`string`) The root folder conta
| 7 | |
| 8 | |
| 9 | class Wikitext2(Dataset): |
| 10 | """Wikitext-2 dataset. |
| 11 | |
| 12 | This is a PyTorch dataset for the pre-processed Wikitext-2 dataset, |
| 13 | containing about 2M words. This dataset will produce vectors of tokens |
| 14 | of fixed size max_length, possibly zero-padded. |
| 15 | |
| 16 | Parameters |
| 17 | ---------- |
| 18 | root (`string`) |
| 19 | The root folder containing the `wiki.*.npz` files. |
| 20 | |
| 21 | split (`string` in (`train`, `validation`, `test`)) |
| 22 | The split to use for the dataset. |
| 23 | |
| 24 | break_mode (`string` in (`none`, `lines`, `complete`)) |
| 25 | The mode for breaking down the dataset into sequences. The different |
| 26 | modes are better explained with an example (with max_length=6) |
| 27 | |
| 28 | Data: |
| 29 | Recurrent Neural Networks |
| 30 | Transformer Networks |
| 31 | Optimization methods for training neural networks |
| 32 | Regularization of neural networks |
| 33 | |
| 34 | - `none` |
| 35 | [Recurrent, Neural, Networks, Transformer, Networks, Optimization] |
| 36 | [methods, for, training, neural, networks, Regularization] |
| 37 | [of, neural, networks, <pad>, <pad>, <pad>] |
| 38 | |
| 39 | - `lines` |
| 40 | [Recurrent, Neural, Networks, <pad>, <pad>, <pad>] |
| 41 | [Transformer, Networks, <pad>, <pad>, <pad>, <pad>] |
| 42 | [Optimization, methods, for, training, neural, networks] |
| 43 | [Regularization, of, neural, networks, <pad>, <pad>] |
| 44 | |
| 45 | - `complete` |
| 46 | [Recurrent, Neural, Networks, Transformer, Networks, <pad>] |
| 47 | [Optimization, methods, for, training, neural, networks] |
| 48 | [Regularization, of, neural, networks, <pad>, <pad>] |
| 49 | |
| 50 | For `lines` and `complete`, lines that are longer than `max_length` |
| 51 | are further broken down into sequences of at most `max_length` (the |
| 52 | process is similar to `break_mode=none`). |
| 53 | |
| 54 | max_length (`int`) |
| 55 | The maximal length of a sequence, and the size of all the vectors |
| 56 | produced by this PyTorch dataset (possibly zero-padded). |
| 57 | |
| 58 | min_length (`int`) |
| 59 | The minimal length of a sequence. Sequences smaller than this number |
| 60 | will be dropped from the dataset. |
| 61 | """ |
| 62 | |
| 63 | def __init__( |
| 64 | self, root, split="train", break_mode="none", max_length=256, min_length=1 |
| 65 | ): |
| 66 |