Neural correlates of affective context in facial expression analysis: A simultaneous EEG-fNIRS study

Document Type

Conference Proceeding

Publication Date

2-23-2016

Abstract

We present a framework to measure the correlation between spontaneous human facial affective expressions and relevant brain activity. The affective states were registered from the video capture of facial expression and related neural activity was measured using wearable and portable neuroimaging systems: functional near infrared spectroscopy (fNIRS), electroencephalography (EEG) to asses both hemodynamic and electrophysiological responses. The methodology involves the simultaneous detection and comparison of various affective expressions by multi-modalities and classification of spatio-temporal data with neural signature traits. The experimental results show strong correlation between the spontaneous facial affective expressions and the affective states related brain activity. We propose a multimodal approach to jointly evaluate fNIRS signals and EEG signals for affective state detection. Results indicate that proposed method with fNIRS+EEG improves performance over fNIRS or EEG only approaches. These findings encourage further studies of the joint utilization of video and brain signals for face perception and brain-computer interface (BCI) applications.

Identifier

84964720814 (Scopus)

ISBN

[9781479975914]

Publication Title

2015 IEEE Global Conference on Signal and Information Processing Globalsip 2015

External Full Text Location

https://doi.org/10.1109/GlobalSIP.2015.7418311

First Page

820

Last Page

824

This document is currently not available here.

Share

COinS