IFAdapter: Instance Feature Control for Grounded Text-to-Image Generation

Yinwei Wu^1,2 Xianpan Zhou¹ Bing Ma¹ Xuefeng Su¹ Kai Ma¹ Xinchao Wang^2†

<sup>1</sup>Tencent PGC&emsp;
<sup>2</sup>National University of Singapore&emsp;
<sup>&dagger;</sup>corresponding author

Overview

We introduce the Instance Feature Adapter(IFAdapter) to to exert fine-grained control over the generation of multiple instances.

Features

The IFAdapter is readily integrated with various community models and LoRAs! We show the results of IFAdapter in combination with PixelArt, Lelo-Lego, Claymation, and Bluepencil. We express our gratitude for these great work contributed by these communities!

Quick Start

Install the required dependencies

git clone https://github.com/WUyinwei-hah/IFAdapter.git

cd IFAdapter

pip install -r requirements.txt

Generation

1. Use the original SDXL for generation

python3 infer.py

2. Use LORAs for generation

python3 infer_loras.py

Citation

@article{wu2024ifadapter,
  title={Ifadapter: Instance feature control for grounded text-to-image generation},
  author={Wu, Yinwei and Zhou, Xianpan and Ma, Bing and Su, Xuefeng and Ma, Kai and Wang, Xinchao},
  journal={arXiv preprint arXiv:2409.08240},
  year={2024}
}