MOGAN: Morphologic-Structure-Aware Generative Learning from a Single Image

Document Type

Article

Publication Date

4-1-2024

Abstract

In most interactive image generation tasks, given regions of interest (ROI) by users, the generated results are expected to have adequate diversities in appearance while maintaining correct and reasonable structures in original images. Such tasks become more challenging if only limited data is available. Recently proposed generative models complete training based on only one image. They pay much attention to the monolithic feature of the sample while ignoring the actual semantic information of different objects inside the sample. As a result, for ROI-based generation tasks, they may produce inappropriate samples with excessive randomicity and without maintaining the related objects' correct structures. To address this issue, this work introduces a morphologic-structure-aware generative adversarial network named MOGAN that produces random samples with diverse appearances and reliable structures based on only one image. For training for ROI, we propose to utilize the data coming from the original image being augmented and bring in a novel module to transform such augmented data into knowledge containing both structures and appearances, thus enhancing the model's comprehension of the sample. To learn the rest areas other than ROI, we employ binary masks to ensure the generation isolated from ROI. Finally, we set parallel and hierarchical branches of the mentioned learning process. Compared with other single image generative adversarial network schemes, our approach focuses on internal features, including the maintenance of rational structures and variation on appearance. Experiments confirm a better capacity of our model on ROI-based image generation tasks than its competitive peers.

Identifier

85179826903 (Scopus)

Publication Title

IEEE Transactions on Systems, Man, and Cybernetics: Systems

External Full Text Location

https://doi.org/10.1109/TSMC.2023.3331227

e-ISSN

21682232

ISSN

21682216

First Page

2021

Last Page

2033

Issue

4

Volume

54

Grant

51775385

Fund Ref

National Natural Science Foundation of China

This document is currently not available here.

Share

COinS