A Domain-Guided Noise-Optimization-Based Inversion Method for Facial Image Manipulation

Document Type

Article

Publication Date

1-1-2021

Abstract

A style-based architecture (StyleGAN2) yields outstanding results in data-driven unconditional generative image modeling. This work proposes a Domain-guided Noise-optimization-based Inversion (DNI) method to perform facial image manipulation. It works based on an inverse code that includes: 1) a novel domain-guided encoder called Image2latent to project the image to StyleGAN2 latent space, which can reconstruct an input image with high-quality and maintain its semantic meaning well; 2) a noise optimization mechanism in which a set of noise vectors are used to capture the high-frequency details such as image edges, further improving image reconstruction quality; and 3) a mask for seamless image fusion and local style migration. We further propose a novel semantic alignment evaluation pipeline. It evaluates the semantic alignment with an inverse code by using different attribute boundaries. Extensive qualitative and quantitative comparisons show that DNI can capture rich semantic information and achieve a satisfactory image reconstruction. It can realize a variety of facial image manipulation tasks and outperform state of the art.

Identifier

85112101743 (Scopus)

Publication Title

IEEE Transactions on Image Processing

External Full Text Location

https://doi.org/10.1109/TIP.2021.3089905

e-ISSN

19410042

ISSN

10577149

PubMed ID

34156940

First Page

6198

Last Page

6211

Volume

30

Grant

61773367

Fund Ref

National Natural Science Foundation of China

This document is currently not available here.

Share

COinS