Zhiliang Peng

About Me

I am a Senior Researcher at Microsoft Research Asia (MSRA), working in the General Artificial Intelligence (GenAI) group. My research interests include multimodal learning and generation, speech and audio intelligence, computer vision, and representation learning.

My recent work includes VibeVoice, a family of open-source models for speech generation and recognition. My earlier research includes Kosmos-2, BEiT, and Conformer.

I conducted my doctoral research at the University of Chinese Academy of Sciences under the supervision of Prof. Qixiang Ye. I received my B.E. degree from Huazhong University of Science and Technology in 2019.

Selected Publications

Full list on Google Scholar

ICLR Oral ★ 52.6K+ GitHub stars 5.9K+ forks

VibeVoice: Expressive Podcast Generation with Next-Token Diffusion

Zhiliang Peng, Jianwei Yu, Wenhui Wang, Yaoyao Chang, Yutao Sun, Li Dong, et al.

The Fourteenth International Conference on Learning Representations (ICLR), 2026

GitHub statistics as of August 2026

Paper Project Code

VIBEVOICE-ASR Technical Report

Zhiliang Peng, Jianwei Yu, Yaoyao Chang, Zilong Wang, Li Dong, et al.

arXiv preprint, 2026

Paper Project Code

VibeVoice-ASR-BitNet Technical Report

Songchen Xu, Ting Song, Shaohan Huang, Zhiliang Peng, Yan Xia, et al.

arXiv preprint, 2026

Paper Code

Multimodal Latent Language Modeling with Next-Token Diffusion

Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, Furu Wei

International Conference on Machine Learning (ICML), 2026

Paper Conference

Kosmos-G multimodal prompt generation examples

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei

International Conference on Learning Representations (ICLR), 2024

Paper arXiv

Kosmos-2: Grounding Multimodal Large Language Models to the World

Zhiliang Peng^*, Wenhui Wang^*, Li Dong^*, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, Furu Wei

International Conference on Learning Representations (ICLR), 2024

Paper Demo Code

Generic-to-Specific Distillation of Masked Autoencoders

Wei Huang^*, Zhiliang Peng^*, Li Dong, Furu Wei, Jianbin Jiao, Qixiang Ye

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

Paper Code

Conformer: Local Features Coupling Global Representations for Recognition and Detection

Zhiliang Peng, Zonghao Guo, Wei Huang, Yaowei Wang, Lingxi Xie, Jianbin Jiao, Qi Tian, Qixiang Ye

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023

Paper Code

Unified masked image modeling paper preview

A Unified View of Masked Image Modeling

Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, Furu Wei

Transactions on Machine Learning Research (TMLR), 2023

Paper Code

Magneto: A Foundation Transformer

Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, et al.

International Conference on Machine Learning (ICML), 2023

Paper arXiv

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks

Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, et al.

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023

Paper Code

Integrally Migrating Pre-trained Transformer Encoder-decoders for Visual Object Detection

Feng Liu, Xiaosong Zhang, Zhiliang Peng, Zonghao Guo, Fang Wan, Xiangyang Ji, Qixiang Ye

IEEE/CVF International Conference on Computer Vision (ICCV), 2023

Paper arXiv

BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers

Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, Furu Wei

arXiv preprint, 2022

Paper Code

Conformer: Local Features Coupling Global Representations for Visual Recognition

Zhiliang Peng, Wei Huang, Shanzhi Gu, Lingxi Xie, Yaowei Wang, Jianbin Jiao, Qixiang Ye

IEEE/CVF International Conference on Computer Vision (ICCV), 2021

Paper Code

TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization

Wei Gao, Fang Wan, Xingjia Pan, Zhiliang Peng, Qi Tian, Zhenjun Han, Bolei Zhou, Qixiang Ye

IEEE/CVF International Conference on Computer Vision (ICCV), 2021

Paper Code

Long-Tailed Distribution Adaptation

Zhiliang Peng, Wei Huang, Zonghao Guo, Xiaosong Zhang, Jianbin Jiao, Qixiang Ye

ACM International Conference on Multimedia (ACM MM), 2021

Paper Code

^* Equal contribution.