Generative Appearance Flow: A Hybrid Approach for Outdoor View Synthesis

    Muhammad Usman Rafique,   Hunter Blanton,   Noah Snavely,   Nathan Jacobs

    Paper   Supplemental   Code   Talk

Overview

Abstract

We address the problem of view synthesis in complex outdoor scenes. We propose a novel convolutional neural network architecture that includes flow-based and direct synthesis sub-networks. Both sub-networks introduce novel elements that greatly improve the quality of the synthesized images. These images are then adaptively fused to create the final output image. Our approach achieves state-of-the-art performance on the KITTI dataset, which is commonly used to evaluate view-synthesis methods. Unlike many recently proposed methods, ours is trained without the need for additional geometric constraints, such as a ground-truth depth map, making it more broadly applicable. Our approach also achieved the best performance on the Brooklyn Panorama Synthesis dataset, which we introduce as a new, challenging benchmark for view synthesis. Our dataset, code, and pretrained models are available at https://mvrl.github.io/GAF.

Recommended citation

@inproceedings{rafique2020gaf,
  title={Generative Appearance Flow: A Hybrid Approach for Outdoor View Synthesis},
  author={Rafique, M. Usman and Blanton, Hunter and Snavely, Noah and Jacobs, Nathan},
  booktitle={The British Machine Vision Conference},
  year={2020}
}