Evaluating and Explaining Natural Language Generation with GenX

Authors

Kayla Duskin, Pacific Northwest National Laboratory
Shivam Sharma, Department of Computer Science
Ji Young Yun, Pacific Northwest National Laboratory
Emily Saldanha, Pacific Northwest National Laboratory
Dustin Arendt, Pacific Northwest National Laboratory

Document Type

Conference Proceeding

Publication Date

1-1-2021

Abstract

Current methods for evaluation of natural language generation models focus on measuring text quality but fail to probe the model creativity, i.e., its ability to generate novel but coherent text sequences not seen in the training corpus. We present the GenX tool which is designed to enable interactive exploration and explanation of natural language generation outputs with a focus on the detection of memorization. We demonstrate the tool on two domain-conditioned generation use cases — phishing emails and ACL abstracts.

Identifier

85123173683 (Scopus)

ISBN

[9781954085398]

Publication Title

Dash La 2021 2nd Workshop on Data Science with Human in the Loop Language Advances Proceedings

First Page

70

Last Page

78

Recommended Citation

Duskin, Kayla; Sharma, Shivam; Yun, Ji Young; Saldanha, Emily; and Arendt, Dustin, "Evaluating and Explaining Natural Language Generation with GenX" (2021). Faculty Publications. 4576.
https://digitalcommons.njit.edu/fac_pubs/4576