Miss Didan Deng

PhD Candidate

The Hong Kong University of Science and Technology

Didan Deng is currently a fifth-year Ph.D. student working under the supervision of Professor Bertram Emil, SHI in the Department of Electronic and Computer Engineering at Hong Kong University of Science and Technology. Her research is focused on solving problems in the current field of automatic emotion recognition technology and providing better machines to recognize emotions accurately and treat groups of people fairly. She is particularly interested in learning emotion recognition with multiple emotion descriptors (e.g., facial action unis, expression categories, valence, etc.). Before coming to HKUST, she graduated from Fudan University in 2017, majored in Electrical Engineering and Automation. She has published five firstauthor conference papers (e.g., ICCV, AAAI) and won three first-place prizes in academic competitions. Her anticipated graduation year is 2022.

Didan Deng’s research is in the field of automatic emotion recognition given human facial inputs. She is interested in several related research topics.

1) Learning a unified emotion model to predict three emotion descriptors, i.e., facial expressions, facial action units, valence, and arousal. The motivation is providing a versatile model to meet the demands of emotion recognition in many applications, such as education, and intelligent robots. The main challenge is to train such a model with incomplete annotations (emotion annotations are very expensive). She proposed a data-driven method to fill in missing labels with iterative knowledge distillation1 . She also proposed a knowledge-aware approach to utilize the relations between multiple emotion descriptors so that regularization can be put during multitask training2 . 2) Learning dynamic features of emotions from visual frames. The motion features of facial muscles are important temporal cues to infer emotions but are sometimes ignored in model design. She proposed a twostream spatial-temporal neural network to learn motion features from two times scales: one scale for micro-expressions, and the other for macroexpressions3 . 3) Learning a multitask emotion model with fair recognition. The emotion models are found to be biased towards some demographic groups of people. This is a problem under-studied, but very critical when emotion recognition is applied for interviews or mental disorder diagnoses. She is studying on improving the multitask emotion model’s fairness towards different genders while preserving its emotion recognition performance. 1 Paper title: Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition. 2 Paper title: Estimating Multiple Emotion Descriptors by Separating Description and Inference. 3 Paper title: Mimamo net: Integrating micro-and macro-motion for video emotion recognition.