Dynamic diffusion transformer

Code

Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang, Yibing Song, Gao Huang, Fan Wang, Yang You

• We propose to dynamically adjust the computation of DiT in different timesteps and spatial locations of images. The computation of DiT-XL could be saved by 50% without sacrificing generation quality.

Dynamic Tuning Towards Parameter and Inference Efficiency for ViT Adaptation

Code

Wangbo Zhao, Jiasheng Tang, Yizeng Han, Yibing Song, Kai Wang, Gao Huang, Fan Wang, Yang You

• We propose to adapt static ViT to dynamic ViT via parameter-efficient fine-tuning without full-parameter tuning.

Mmbench: Is your multi-modal model an all-around player?

Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, Dahua Lin

• We propose MMBench, a bilingual benchmark for assessing the multi-modal capabilities of VLMs.

VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning

Ziyang Luo, Nian Liu, Wangbo Zhao, Xuguang Yang, Dingwen Zhang, Deng-Ping Fan, Fahad Khan, Junwei Han

• We introduce VSCode a generalist model with novel 2D prompt learning to jointly address four SOD tasks and three COD tasks

Multi-grained temporal prototype learning for few-shot video object segmentation

Nian Liu, Kepan Nan, Wangbo Zhao, Yuanwei Liu, Xiwen Yao, Salman Khan, Hisham Cholakkal, Rao Muhammad Anwer, Junwei Han, Fahad Shahbaz Khan

• We propose to leverage multi-grained temporal guidance information for handling the temporal correlation nature of video data for few-shot video object segmentation

Modeling motion with multi-modal features for text-based video segmentation

Wangbo Zhao, Kai Wang, Xiangxiang Chu, Fuzhao Xue, Xinchao Wang, Yang You

• We design a method to fuse and align appearance, motion, and linguistic features to achieve accurate text-based video segmentation.

Light field saliency detection with dual local graph learning and reciprocative guidance

Nian Liu, Wangbo Zhao, Dingwen Zhang, Junwei Han, Ling Shao

• We introduce a reciprocative guidance scheme for light field saliency detection.

Weakly supervised video salient object detection

Wangbo Zhao, Jing Zhang, Long Li, Nick Barnes, Nian Liu, Junwei Han

• We present the first weakly supervised video salient object detection model based on relabeled fixation guided scribble annotations.