Tianyu He

Tianyu He

Senior Researcher

Microsoft Research Asia

Research Interests

Multimodal Foundation Models
Interactive Video World Models

About

I am a Tech Lead at XYZ AI Lab, where I lead the lab's efforts in multimodal AI. Previously, I was a Senior Researcher in the Machine Learning Group at Microsoft Research Asia. My work focuses on the intersection of multimodality and generative modeling, specifically aimed at advancing how AI understands and creates complex, real-world content.

I graduated from the University of Science and Technology of China (USTC) in 2019. Prior to joining Microsoft in 2022, I spent three years at Alibaba DAMO Academy (Talent Program) developing industry-leading algorithms. My research is regularly published in top-tier venues such as NeurIPS, ICLR, CVPR, ICCV, and ECCV.

Driven by a long-term goal of building intelligence for the physical world, my current research focuses on:

  • Multimodal Foundation Models: Engineering unified architectures with a focus on data/model scaling and reasoning to power next-generation agents and robotics.
  • Interactive Video World Models: Exploring real-time causal generation, interactive control, and consistent world modeling to bridge the gap between digital synthesis and physical simulation.

๐Ÿš€ I am hiring! If you are interested in a position focused on Multimodal Foundation Models or World Models! Please email me if you are interested.

Selected Projects

View All โ†’

SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, Yukang Chen, Xinting Hu, Tianyu He, Shaoshuai Shi, Zhuotao Tian, Xin Wang, Mike Zheng Shou, Li Jiang

arXiv preprint arXiv:2609.02886 (2026)

A fully open foundation for building interactive video world models.

AI4AI at Scale: A Full-Pipeline System for Enhancing LLM Agentic Capabilities

XYZ Agentic Team

Technical Report (2026)

Two open-weight Deep Search agents: XYZ-Aquila-mini (35B) and XYZ-Aquila-pro (397B), achieving state-of-the-art results across multiple Deep Search benchmarks.

Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling

Haoyu Wu, Diankun Wu, Tianyu He, Junliang Guo, Yang Ye, Yueqi Duan, Jiang Bian

International Conference on Learning Representations (ICLR) (2026)

Combining video diffusion with 3D representation for geometrically consistent world modeling.

Reinforcement Learning with Inverse Rewards for World Model Post-training

Yang Ye, Tianyu He, Shuo Yang, Jiang Bian

arXiv preprint arXiv:2509.23958 (2025)

A post-training framework that derives verifiable reward signals by recovering input actions from generated videos using an Inverse Dynamics Model.

Playing with Transformer at 30+ FPS via Next-Frame Diffusion

Xinle Cheng, Tianyu He, Jiayi Xu, Junliang Guo, Di He, Jiang Bian

arXiv preprint arXiv:2506.01380 (2025)

Achieving autoregressive video generation at 30+ FPS through next-frame diffusion.

MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft

Junliang Guo, Yang Ye, Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, Jiang Bian

arXiv preprint arXiv:2504.08388 (2025)

A real-time and open-source interactive world model built on Minecraft.

Video in-context Learning: Autoregressive Transformers are Zero-Shot Video Imitators

Wentao Zhang, Junliang Guo, Tianyu He, Li Zhao, Linli Xu, Jiang Bian

International Conference on Learning Representations (ICLR) (2025)

Demonstrating that autoregressive transformers can perform zero-shot video imitation via in-context learning.

VidTok: A Versatile and Open-Source Video Tokenizer

Anni Tang, Tianyu He, Junliang Guo, Xinle Cheng, Li Song, Jiang Bian

arXiv preprint arXiv:2412.13061 (2024)

A versatile and open-source video tokenizer for video understanding and generation. It has been integrated into the Hugging Face Diffusers library (34K GitHub stars).

IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI

Xiaoyu Chen, Junliang Guo, Tianyu He, Chuheng Zhang, Pushi Zhang, Derek Cathera Yang, Li Zhao, Jiang Bian

arXiv preprint arXiv:2411.00785 (2024)

Proposing latent action models for embodied AI foundation models.

GAIA: Zero-Shot Talking Avatar Generation

Tianyu He, Junliang Guo, Runyi Yu, Yuchi Wang, Jialiang Zhu, Kaikai An, Leyi Li, Xu Tan, Chunyu Wang, Han Hu, HsiangTao Wu, Sheng Zhao, Jiang Bian

International Conference on Learning Representations (ICLR) (2024)

Zero-shot talking avatar generation without subject-specific fine-tuning.