Awesome-Open-Vision-Problem-Modeling

⭐Practical New Tasks and Inspiring Modeling Solutions for Diverse Open Vision Problems⭐

2025 | MDD | ICCV

MDD: A Dataset for Text-and-Music Conditioned Duet Dance Generation
^{Prerit Gupta, Jason Alexander Fotso-Puepi, Zhengyuan Li, Jay Mehta, Aniket Bera}
Paper/Code

Task: Text-to-Duet for generating coordinated partner movements from a text description and music, and Text-to-Dance Accompaniment for synthesizing context-aware follower motion given the leader’s motion, music and text description.	Modeling: MDM/InterGen/Duolando

### 2025 | NRCD | IGARSS Non-Registration Change Detection: A Novel Change Detection Task and Benchmark Dataset
^{Zhe Shan, Lei Zhou, Liu Mao, Shaofan Chen, Chuanqiu Ren, Xia Xie}
Paper/Code

Task: Non-registration change detection.	Modeling: General change detection models.

2025 | MOSPA | arXiv

MOSPA: Human Motion Generation Driven by Spatial Audio
^{Shuyang Xu, Zhiyang Dou, Mingyi Shi, Liang Pan, Leo Ho, Jingbo Wang, Yuan Liu, Cheng Lin, Yuexin Ma, Wenping Wang, Taku Komura}
Paper

Task: Human motion generation task centered on spatial audio-driven human motion synthesis.	Modeling: A diffusion-based generative framework (MOSPA) captures the relationship between body motion and spatial audio through
an effective fusion mechanism.

2025 | EMind | arXiv

EMind: A Foundation Model for Multi-task Electromagnetic Signals Understanding
^{Luqing Luo, Wenjin Gui, Yunfei Liu, Ziyue Zhang, Yunxi Zhang, Fengxiang Wang, Zonghao Guo, Zizhi Ma, Xinzhu Liu, Hanxiang He, Jinhai Li, Xin Qiu, Wupeng Xie, Yangang Sun}
Paper/Code

Task: Multi-task Electromagnetic Signals Understanding.	Modeling: EMind, an electromagnetic signals foundation model that bridges large scale pretraining and the unique nature of this modality based on the proposed first unified and largest standardized electromagnetic signal dataset covering multiple signal types and tasks.

2025 | GraphDoc | ICLR

Graph-based Document Structure Analysis
^{Yufan Chen, Ruiping Liu, Junwei Zheng, Di Wen, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen}
Paper/Code

Task: Graph-based Document Structure Analysis.	Modeling: Document Relation Graph Generator(DRGG) combines object detection with relation prediction, capturing both spatial and logical relatioins between document elements.

2025 | Paper2Poster | arXiv

Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers
^{Wei Pang, Kevin Qinghong Lin, Xiangru Jian, Xi He, Philip Torr}
Paper/Code

Task: Automatic academic poster generation from long-context scientific papers.	Modeling: PosterAgent, a visual-in-the-loop multi-agent pipeline with parsing, binary-tree layout planning, and iterative VLM-guided rendering refinement.

2025 | DiffSensei | CVPR

DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation
^{Jianzong Wu, Chao Tang, Jingbo Wang, Yanhong Zeng, Xiangtai Li, Yunhai Tong}
Paper/Code

Task: Customized manga generation with dynamic multi-character control from text.	Modeling: Diffusion–MLLM hybrid framework using a text-compatible identity adapter and masked cross-attention for precise character and layout control.

2025 | LLada | ICCV

Where, What, Why: Towards Explainable Driver Attention Prediction
^{Yuchen Zhou, Jiayu Tang, Xiaoyan Xiao, Yueyao Lin, Linkai Liu, Zipeng Guo, Hao Fei, Xiaobo Xia, Chao Gou}
Paper/Code

Task: Explainable Driver Attention Prediction that identifies where drivers look, what they attend to, and why.	Modeling: LLada, an LLM-driven framework integrating pixel-level attention, semantic parsing, and causal reasoning in an end-to-end model.

2025 | DiffHDR | AAAI

Predicting the Original Appearance of Damaged Historical Documents
^{Zhenhua Yang, Dezhi Peng, Yongxin Shi, Yuyi Zhang, Chongyu Liu, Lianwen Jin}
Paper/Code

Task: Historical Document Repair to reconstruct the original appearance of damaged historical texts.	Modeling: Diffusion-based DiffHDR with semantic–spatial guidance and character perceptual loss for coherent restoration.

2025 | Rose4D | CVPR

Birth and Death of a Rose
^{Chen Geng, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu}
Paper

Task: Generation of temporally evolving 3D object intrinsics from pretrained 2D models.	Modeling: Neural Template–guided temporal-state distillation from 2D diffusion features for consistent multi-view, multi-time rendering.

2025 | GFlow | AAAI

GFlow: Recovering 4D World from Monocular Video
^{Shizun Wang, Xingyi Yang, Qiuhong Shen, Zhenxiang Jiang, Xinchao Wang}
Paper/Code

Task: 4D scene reconstruction from a single monocular video without camera parameters.	Modeling: 2D-prior-guided 4D Gaussian flow with alternating pose and dynamics optimization.

2025 | VecFormer | arXiv

Point or Line? Using Line-based Representation for Panoptic Symbol Spotting in CAD Drawings
^{Xingguang Wei∗, Haomin Wang∗, Shenglong Ye, Ruifeng Luo, Yanting Zhang, Lixin Gu, Jifeng Dai, Yu Qiao, Wenhai Wang, Hongjie Zhang}
Paper

Task: Computer-aided design (CAD) drawings.	Modeling: Type-agnostic and expressive line-based representation.

2025 | SimGen | arXiv

SIMGEN: A DIFFUSION-BASED FRAMEWORK FOR SIMULTANEOUS SURGICAL IMAGE AND SEGMENTATION MASK GENERATION
^{Aditya Bhat, Rupak Bose, Chinedu Innocent Nwoye, Nicolas Padoy}
Paper

Task: Simultaneous generation of surgical images and corresponding segmentation masks.	Modeling: Diffusion-based framework with cross-correlation priors and Canonical Fibonacci Lattice for aligned image–mask synthesis.

2024 | PanAf20K | IJCV

PanAf20K: A Large Video Dataset for Wild Ape Detection and Behaviour Recognition
^{Otto Brookes, Majid Mirmehdi, Colleen Stephens, Samuel Angedakin, Katherine Corogenes, Dervla Dowd, Paula Dieguez, Thurston C. Hicks, Sorrel Jones, Kevin Lee, Vera Leinert, Juan Lapuente, Maureen S. McCarthy, Amelia Meier, Mizuki Murai, Emmanuelle Normand, Virginie Vergnes, Erin G. Wessling, Roman M. Wittig, Kevin Langergraber, Nuria Maldonado, Xinyu Yang, Klaus Zuberb¨uhler, Christophe Boesch, Mimi Arandjelovic, Hjalmar K¨uhl, Tilo Burghardt}
Paper/Code

Task: Wild Ape Detection and Behaviour Recognition.	Modeling: Object Detection + Behavioural Action Recognition.

2024 | NIGHTTIME OPTICAL FLOW | ICLR

EXPLORING THE COMMON APPEARANCE-BOUNDARY ADAPTATION FOR NIGHTTIME OPTICAL FLOW
^{Hanyu Zhou, Yi Chang, Haoyue Liu, Wending Yan, Yuxing Duan, Zhiwei Shi, Luxin Yan}
Paper

Task Definition: Nighttime optical flow estimation.	Modeling: Cross-domain motion alignment.

2024 | OVCoser | ECCV

Open-Vocabulary Camouflaged Object Segmentation
^{Youwei Pang, Xiaoqi Zhao, Jiaming Zuo, Lihe Zhang, Huchuan Lu}
Paper/Code

Task: open-vocabulary camouflaged object segmentation.	Modeling: open-set recognition and camouflage-aware segmentation.