Header logo is

Learning Physics-guided Face Relighting under Directional Light

2020

Conference Paper

ps


Relighting is an essential step in realistically transferring objects from a captured image into another environment. For example, authentic telepresence in Augmented Reality requires faces to be displayed and relit consistent with the observer's scene lighting. We investigate end-to-end deep learning architectures that both de-light and relight an image of a human face. Our model decomposes the input image into intrinsic components according to a diffuse physics-based image formation model. We enable non-diffuse effects including cast shadows and specular highlights by predicting a residual correction to the diffuse render. To train and evaluate our model, we collected a portrait database of 21 subjects with various expressions and poses. Each sample is captured in a controlled light stage setup with 32 individual light sources. Our method creates precise and believable relighting results and generalizes to complex illumination conditions and challenging poses, including when the subject is not looking straight at the camera.

Author(s): Nestmeyer, Thomas and Lalonde, Jean-François and Matthews, Iain and Lehrmann, Andreas M
Book Title: Conference on Computer Vision and Pattern Recognition
Pages: 5123--5132
Year: 2020
Month: June
Publisher: IEEE/CVF

Department(s): Perceiving Systems
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Place: Seattle

State: Accepted

Links: Paper
Attachments:

BibTex

@inproceedings{nestmeyer2020faceRelighting,
  title = {Learning Physics-guided Face Relighting under Directional Light},
  author = {Nestmeyer, Thomas and Lalonde, Jean-François and Matthews, Iain and Lehrmann, Andreas M},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  pages = {5123--5132},
  publisher = {IEEE/CVF},
  month = jun,
  year = {2020},
  month_numeric = {6}
}