Multi-task Recommendation in PyTorch

Introduction

MTReclib provides a PyTorch implementation of multi-task recommendation models and common datasets. Currently, we implmented 7 multi-task recommendation models to enable fair comparison and boost the development of multi-task recommendation algorithms. The currently supported algorithms include: * SingleTask：Train one model for each task, respectively * Shared-Bottom: It is a traditional multi-task model with a shared bottom and multiple towers. * OMoE: Adaptive Mixtures of Local Experts (Neural Computation 1991) * MMoE: Modeling Task Relationships in Multi-task Learning with Multi-Gate Mixture-of-Experts (KDD 2018) * PLE: Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations (RecSys 2020 best paper) * AITM: Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising (KDD 2021) * MetaHeac: Learning to Expand Audience via Meta Hybrid Experts and Critics for Recommendation and Advertising (KDD 2021)

Datasets

• AliExpressDataset: This is a dataset gathered from real-world traffic logs of the search system in AliExpress. This dataset is collected from 5 countries: Russia, Spain, French, Netherlands, and America, which can utilized as 5 multi-task datasets. Original_dataset Processed_dataset Google Drive Processed_dataset Baidu Netdisk

For the processed dataset, you should directly put the dataset in './data/' and unpack it. For the original dataset, you should put it in './data/' and run 'python preprocess.py --dataset_name NL'.

Requirements

• Python 3.6
• PyTorch > 1.10
• pandas
• numpy
• tqdm

Run

Parameter Configuration:

• dataset_name: choose a dataset in ['AliExpress_NL', 'AliExpress_FR', 'AliExpress_ES', 'AliExpress_US'], default for AliExpress_NL
• dataset_path: default for ./data
• model_name: choose a model in ['singletask', 'sharedbottom', 'omoe', 'mmoe', 'ple', 'aitm', 'metaheac'], default for metaheac
• epoch: the number of epochs for training, default for 50
• task_num: the number of tasks, default for 2 (CTR & CVR)
• expert_num: the number of experts for ['omoe', 'mmoe', 'ple', 'metaheac'], default for 8
• learning_rate: default for 0.001
• batch_size: default for 2048
• weight_decay: default for 1e-6
• device: the device to run the code, default for cuda:0
• save_dir: the folder to save parameters, default for chkpt

You can run a model through:

python main.py --model_name metaheac --num_expert 8 --dataset_name AliExpress_NL

Results

For fair comparisons, the learning rate is 0.001, the dimension of embeddings is 128, and mini-batch size is 2048 equally for all models. We report the mean AUC and Logloss over five random runs. Best results are in boldface.

File Structure

.
├── main.py
├── README.md
├── models
│   ├── layers.py
│   ├── aitm.py
│   ├── omoe.py
│   ├── mmoe.py
│   ├── metaheac.py
│   ├── ple.py
│   ├── singletask.py
│   └── sharedbottom.py
└── data
    ├── preprocess.py         # Preprocess the original data
    ├── AliExpress_NL         # AliExpressDataset from Netherlands
        ├── train.csv
    └── test.py
    ├── AliExpress_ES         # AliExpressDataset from Spain
    ├── AliExpress_FR         # AliExpressDataset from French
    └── AliExpress_US         # AliExpressDataset from America

Contact

If you have any problem about this library, please create an issue or send us an Email at: * zhuyc0204@gmail.com * zhuangfuzhen@buaa.edu.cn

Reference

If you use this repository, please cite the following papers:

@inproceedings{zhu2021learning,
  title={Learning to Expand Audience via Meta Hybrid Experts and Critics for Recommendation and Advertising},
  author={Zhu, Yongchun and Liu, Yudan and Xie, Ruobing and Zhuang, Fuzhen and Hao, Xiaobo and Ge, Kaikai and Zhang, Xu and Lin, Leyu and Cao, Juan},
  booktitle={Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining},
  pages={4005--4013},
  year={2021}
}

@inproceedings{xi2021modeling,
  title={Modeling the sequential dependence among audience multi-step conversions with multi-task learning in targeted display advertising},
  author={Xi, Dongbo and Chen, Zhen and Yan, Peng and Zhang, Yinger and Zhu, Yongchun and Zhuang, Fuzhen and Chen, Yu},
  booktitle={Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining},
  pages={3745--3755},
  year={2021}
}

Some model implementations and util functions refers to these nice repositories.

• pytorch-fm: This package provides a PyTorch implementation of factorization machine models and common datasets in CTR prediction.
• MetaHeac: This is an official implementation for Learning to Expand Audience via Meta Hybrid Experts and Critics for Recommendation and Advertising.