Config load from json file

One layer rnn.

Multi-Head Attention module

Reference: Attention is all you need Input shape is: [batch, seq_len, dim_k] and [batch, seq_len, dim_k] [batch, seq_len,

Softmax focal loss references: Focal Loss for Dense Object Detection https://github.com/Hsuxu/FocalLoss-PyTorch

Reference: A New Method of Region Embedding for Text Classification

Input shape is: [batch, seq_len, dim] and [batch, seq_len, dim] Output is same with the first input

Implements BERT version of Adam algorithm with weight decay fix. Params: lr: learning rate warmup: portion of t_total for the warm

Compose with two layers

Reference: Highway Networks. For now we don't limit the type of the gate and forward. Caller should init Highway with transformer and car

Reference: attention is all you need

A two-feed-forward-layer module

Scaled Dot-Product Attention

Star-Transformer: https://arxiv.org/pdf/1902.09113v2.pdf

Reference: Hierarchical Attention Networks for Document Classification

Standard names for activation type

Standard names for activation

Also known as Soft Attention or Bahdanau Attention Reference: Neural machine translation by jointly learning to align and translate

Input shape is: [batch, dim] and [batch, seq_len, dim] Output is same with the first input

Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms

Reference: Deep Pyramid Convolutional Neural Networks for Text Categorization

Base dataset class

Standard names for doc embedding type.

Standard names for embedding mode Given the vocab tensor shape[batch_size, sequence_len]. The following keys are defined: * `FLAT`: Normal

Standard names for embedding type The following keys are defined: * `EMBEDDING`: Return the embedding after lookup. * `REGION_EMBEDDING`:

Standard names for fan mode

Implement fasttext classification method Reference: "Bag of Tricks for Efficient Text Classification"

FastText Collator Extra support features: token, token-ngrams, keywords, topics.

Implement HMCN(Hierarchical Multi-Label Classification Networks) Reference: "Hierarchical Multi-Label Classification Networks"

Standard names for init

Standard names for embedding mode Given the vocab tensor shape[batch_size, sequence_len]. The following keys are defined: * `FLAT`: Normal

Standard names for loss type

Standard names for model modes. The following standard keys are defined: * `TRAIN`: training mode. * `EVAL`: evaluation mode. * `PREDI

Reference: Attention is all you need

Standard names for optimizer

Implement region embedding classification method Reference: "A New Method of Region Embedding for Text Classification"

Standard names for region embedding type

TextRNN + TextCNN

Implement TextRNN, contains LSTM，BiLSTM，GRU，BiGRU Reference: "Effective LSTMs for Target-Dependent Sentiment Classification" "Bidir