Tianfan Xue

 

Tianfan Xue
Vice Chancellor Assistant Professor
the Department of Information Engineering
the Chinese University of Hong Kong
Email: loading... | loading...
Note: tfxue@mit.edu and tianfan@google.com are no longer active, please contact me through emails above
Google scholar | Linkedin

Room 717 SHB
The Chinese University of Hong Kong,
Shatin, N.T.
Hong Kong

Biography

I started as an Vice Chancellor assistant professor at the Department of Information Engineering, the Chinese University of Hong Kong in Novermber of 2022. If you are excited about computational photography, computer vision, computer graphics, and machine learning, please contact for potential full-funded Ph.D., master, postdoc, R.A., and intern positions available.

I received my Ph.D. from MIT Computer Science and Artificial Intelligence Laboratory, under the supervision of William T. Freeman. Before that, I received my Master of Philosophy in the Information Engineering Department, The Chinese University of Hong Kong in 2011, under the supervised of Prof. Xiaoou Tang and Prof. Jianzhuang Liu. I also received my B.E. degree in the Computer Science and Technology Department from Tsinghua University, China, in 2009. Here is my Curriculum Vitae [PDF].

Research Highlights

Lab members


Xiao Chen (陈骁)
Ph.D. candidate

Xin Cai (蔡昕)
Ph.D. candidate

Lihe Ding (丁立鹤)
Ph.D. candidate

Zhiyuan You (尤志远)
Ph.D. candidate
(Co- with Chao Dong)

Xiao Fu (付潇)
Ph.D. candidate
(Co- with Dahua Lin)

Hongyu Deng (邓宏宇)
Ph.D. candidate
(Co- with He Chen)

Visiting/Collaborated Students

Selected Publications


 

Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models

Zhiyuan You*, Zheyuan Li*, Jinjin Gu*, Zhenfei Yin, Tianfan Xue, Chao Dong
arXiv 2312.08962, 2024. [PDF] [Webpage]

 

Event-Based Motion Magnification

Yutian Chen*, Shi Guo*, Fangzheng Yu, Feng Zhang, Jinwei Gu, Tianfan Xue
arXiv 2402.11957, 2024. [PDF] [Webpage] [Code] [YouTube] [Bilibili]

 

GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction

Xiao Chen, Quanyi Li, Tai Wang, Tianfan Xue, Jiangmiao Pang
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage] [YouTube]

 

Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priors

Lihe Ding*, Shaocong Dong*, Zhanpeng Huang, Zibin Wang, Yiyuan Zhang, Kaixiong Gong, Dan Xu, Tianfan Xue
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage]

 

Interactive3D: Create What You Want by Interactive 3D Generation

Shaocong Dong*, Lihe Ding*, Zhanpeng Huang, Zibin Wang, Tianfan Xue, Dan Xu
Computer Vision and Pattern Recognition (CVPR), 2024.

 

HDRFlow: Real-Time HDR Video Reconstruction with Large Motions

Gangwei Xu*, Yujin Wang*, Jinwei Gu, Tianfan Xue, Xin Yang
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage]

 

EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI

Tai Wang*, Xiaohan Mao*, Chenming Zhu*, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, Xihui Liu, Cewu Lu, Dahua Lin, Jiangmiao Pang
Computer Vision and Pattern Recognition (CVPR), 2024. [PDF] [Webpage] [Code&Data]

 

Obj-NeRF: Extracting Object NeRFs from Multi-view Images

Zhiyi Li, Lihe Ding, Tianfan Xue
arXiv 2311.15291, 2023. [PDF] [Webpage]

 

AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training

Yifan Jiang, Peter Hedman, Ben Mildenhall, Dejia Xu, Jonathan T. Barron, Zhangyang Wang, Tianfan Xue
Computer Vision and Pattern Recognition (CVPR), 2023. [PDF] [Webpage]

 

Fast and High-Quality Image Denoising via Malleable Convolutions

Yifan Jiang, Bartlomiej Wronski, Ben Mildenhall, Jonathan T. Barron, Zhangyang Wang, Tianfan Xue
European Conference on Computer Vision (ECCV), 2022. [PDF] [Webpage]

 

How to Train Neural Networks for Flare Removal

Yicheng Wu, Qiurui He, Tianfan Xue, Rahul Garg, Jiawen Chen, Ashok Veeraraghavan, Jonathan T. Barron
International Conference on Computer Vision ICCV, 2021. [PDF] [Webpage]

 

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Shumian Xin, Neal Wadhwa, Tianfan Xue, Jonathan T. Barron, Pratul P. Srinivasan, Jiawen Chen, Ioannis Gkioulekas, Rahul Garg
International Conference on Computer Vision ICCV, 2021. [PDF] [Code]

 

Learned Dual-View Reflection Removal

Simon Niklaus, Xuaner Cecilia Zhang, Jonathan T. Barron, Neal Wadhwa, Rahul Garg, Feng Liu, Tianfan Xue
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021. [PDF] [Webpage]

 

Joint Bilateral Learning for Real-time Universal Photorealistic Style Transfer

Xide Xia, Meng Zhang, Tianfan Xue, Zheng Sun, Hui Fang, Brian Kulis, Jiawen Chen
European Conference on Computer Vision (ECCV), 2020. [arxiv-preprint] [YouTube] [Dataset]

Real-time Localized Photorealistic Video Style Transfer.

Xide Xia, Tianfan Xue, Wei-sheng Lai, Zheng Sun, Abby Chang, Brian Kulis, Jiawen Chen, 2020.
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021. [PDF]

 

Handheld Mobile Photography in Very Low Light

Orly Liba, Kiran Murthy, Yun-Ta Tsai, Timothy Brooks, Tianfan Xue, Nikhil Karnad, Qiurui He, Jonathan T. Barron, Dillon Sharlet, Ryan Geiss, Samuel W. Hasinoff, Yael Pritch, Marc Levoy
ACM SIGGRAPH Asia, 2019, Oral
[PDF] [Webpage]

 

Multi-frame stereo matching with edges, planes, and superpixels,

Tianfan Xue, Andrew Owens, Daniel Scharstein, Michael Goesele, Richard Szeliski
Image and Vision Computing (IVC), 2019
[PDF] [Webpage]

 

Unprocessing Images for Learned Raw Denoising,

Tim Brooks, Ben Mildenhall, Tianfan Xue, Jiawen Chen, Dillon Sharlet, Jonathan T. Barron
Computer Vision and Pattern Recognition (CVPR), 2019
[PDF] [Website] [github]

 

Stereoscopic Dark Flash for Low-light Photography,

Jian Wang, Tianfan Xue, Jonathan T. Barron, Jiawen Chen
International Conference on Computational Photography (ICCP), 2019
[PDF] [YouTube]

 

MoSculp: Interactive Visualization of Shape and Time,

Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman
User Interface Software and Technology (UIST), 2018
[PDF] [Website] [github]
Press Coverage: Forbes, BBC, MIT (featured as the 9/19 homepage), etc.

 

Video Enhancement with Task-Oriented Flow,

Tianfan Xue, Baian Chen, Jiajun Wu, Donglai Wei, William T. Freeman
International Journal of Computer Vision (IJCV), 2018
[PDF] [arxiv-preprint] [Website] [github] [Dataset]

 

Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling

Xingyuan Sun*, Jiajun Wu*, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B. Tenenbaum, and William T. Freeman
Computer Vision and Pattern Recognition (CVPR), 2018
[PDF] [Website] [Dataset] [Github]

 

Best-Buddies Similarity - Robust Template Matching using Mutual Nearest Neighbors,

Shaul Oron, Tali Dekel, Tianfan Xue, William T. Freeman, Shai Avidan,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2018
[PDF] [arxiv-preprint]

 

MarrNet: 3D Shape Reconstruction via 2.5D Sketches

Jiajun Wu*, Yifan Wang*, Tianfan Xue, Xingyuan Sun, William T. Freeman, and Joshua B. Tenenbaum
Neural Information Processing Systems (NIPS), 2017
[PDF] [Website]

 

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, and William T. Freeman,
Neural Information Processing Systems (NIPS) 2016, Oral.

Visual dynamics: Stochastic future generation via layered cross convolutional networks

Tianfan Xue*, Jiajun Wu*, Katherine L. Bouman, and William T. Freeman,
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI) 2018.
[PDF (conference)] [PDF (journal)] [arxiv-preprint] [Website] [Video] [YouTube] [Code(Github)]

 

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Jiajun Wu*, Chengkai Zhang*, Tianfan Xue, William T. Freeman, and Joshua B. Tenenbaum,
Neural Information Processing Systems (NIPS) 2016.
[PDF] [arxiv-preprint] [Website] [YouTube] [Code(Github)]

 

Single Image 3D Interpreter Network

Jiajun Wu*, Tianfan Xue*, Joseph J. Lim, Yuandong Tian, Joshua B. Tenenbaum, Antonio Torralba, and William T. Freeman,
European Conference on Computer Vision (ECCV), 2016, Oral.
[PDF] [Website] [IKEA-dataset] [Keypoint5-dataset] [arxiv-preprint] [Code(Github)]

3D Interpreter Networks for Viewer-Centered Wireframe Modeling

International Journal of Computer Vision (IJCV), 2018. [arxiv-preprint]

 

A Computational Approach for Obstruction-Free Photography

Tianfan Xue, Michael Rubinstein, Ce Liu, William T. Freeman,
ACM SIGGRAPH 2015, Oral.
[PDF] [Youtube] [Website]

 

The Aperture Problem for Refractive Motion

Tianfan Xue, Hossein Mobahi, Fredo Durand, William T. Freeman,
IEEE Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2015.
[PDF] [Poster] [Video] [Website]

 

Refraction Wiggles for Measuring Fluid Depth and Velocity from Video

Tianfan Xue, Michael Rubinstein, Neal Wadhwa, Anet Levin, Fredo Durand, William T. Freeman,
European Conference on Computer Vision (ECCV) 2014, Oral.
[PDF] [Slides] [Data] [Website] [Presentation] [Code]

 

Example-Based 3D Object Reconstruction from Line Drawings

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang,
IEEE Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2012.
[PDF] [Poster] [Dataset]

 

Symmetric Piecewise Planar Object Reconstruction from a Single Image

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang, (CVPR) 2011.[PDF]

3D Modeling from a Single View of a Symmetric Object

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang, Transactions on Image Processing (TIP) 2012. [PDF]

 

Object Cut: Complex 3D object reconstruction through line drawing separation

Tianfan Xue, Jianzhuang Liu, Xiaoou Tang,
Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2010. [PDF]

 

Joint Example-based DepthMap Super-Resolution

Yanjie Li, Tianfan Xue, Lifeng Sun, Jianzhuang Liu,
International Conference on Multimedia Expo (ICME) 2012. [PDF] [Code]

 

Fast Frame-rate Up-conversion of Depth Video via Video Coding

Yanjie Li, Lifeng Sun, Tianfan Xue,
ACM Multimedia (MM) 2011. [PDF]

* indicates equal contribution

Education

Services

MISC