A Review of Convolutional Neural Network Model for Audio-Visual Features Extraction in Personality Traits Recognition

Abstract

In the field of personality computing research, the analysis on audio-visual input is predominantly used to detect human personality behaviors. With the advancement of computer vision technology, there has been significant enhancement in personality computing. Personality trait recognition is one of the applications under personality computing where the machine can analyze human behaviors and recognize personality traits via video analysis. In video input, there are different audio-visual features characteristics, consisting of visual (images) and audio (sounds) elements. Therefore, it is critical to employ appropriate deep learning techniques to most effectively extract important features from audio-visual input. The maturity of convolutional neural networks (CNNs) has been proven with promising prediction results for feature extraction and selection in image classification, sound detection, and face-emotion recognition. Thus, a variety of CNN-based techniques have been developed with different salient features and CNN layer modifications to learn and extract meaningful patterns and representations from images and videos. Due to the distinct characteristics of audio-visual features, hybrid CNN-based techniques were introduced to optimally analyze these modalities. This study aimed to explore hybrid CNN-based techniques used in video analysis for personality trait recognition systems. This study also provides an overview of current issues in the development of recognition models in personality computing using hybrid CNN-based techniques. The advantages of integrating audio and visual modalities in hybrid techniques are addressed, as well as their performance accuracy. The discussion finally summarizes the findings and potential future research directions.