R. Gnana Praveen

I am a post-doctoral researcher at the Computer Research Institute of Montreal (CRIM), working on audio-visual learning for person verification. I did PhD in artificial intelligence (focused on computer vision and affective computing) at LIVIA lab, ETS Montreal, Canada under the supervision of Prof. Eric Granger and Prof. Patrick Cardinal in 2023. In my thesis, I have worked on developing weakly supervised learning (multiple instance learning) models for facial expression recognition in videos and novel attention models for audio-visual fusion in dimensional emotion recognition.

Before my PhD, I had 5 years of industrial research experience in computer vision, working for giant companies as well as start-ups including Samsung Research India, Synechron India and upGradCampus India. I also had the privilege of working with Prof. R. Venkatesh Babu at Indian Institute of Science, Bangalore on crowd flow analysis in videos. I did my Masters at Indian Institute of Technology Guwahati under the supervision of Prof. Kannan Karthik in 2012.

I like to play rhythm instruments in my free time. I also prefer to read books and occasionally do blogging. Here is the collection of my musings.

Email  /  CV  /  Google Scholar  /  Twitter  /  ResearchGate  /  Github  /  LinkedIn

• ^New!! Apr 2024: Our work on "Recursive Joint Cross-modal attention" has been accepted at ABAW@CVPR 2024 Workshop
• Mar 2024: Achieved 2nd place in the valence-arousal challenge of 6th ABAW@CVPR 2024 competition
• Mar 2024: Our research article on joint cross-attention for audio-visual fusion has been featured in March issue of IEEE Biometrics Newsletter.
• Mar 2024: One paper accepted at ICME 2024 (CORE A)!
• Mar 2024: Two papers accepted at FG 2024!
• Oct 2023: Paper accepted at NeurIPS 2023 3rd workshop on ENLSP
• Sep 2023: Serving as a reviewer for WACV 2024
• Jun 2023: Serving as a reviewer for ACM MM 2023
• Jun 2023: Presented our work "Recurrent Joint Attention for Audio-Visual Fusion in Regression-based Emotion Recognition" at ICASSP 2023
• May 2023: Successfully defended my Ph.D. Thesis titled "Deep Regression Models for Spatiotemporal Expression Recognition in Videos"
• Mar 2023: Started working as a post-doctoral researcher at Computer Research Institute of Montreal (CRIM)"

I'm interested in computer vision, affective computing, deep learning, and multimodal video understanding models. Most of my research revolves around video analytics, weakly supervised learning, facial behavior analysis, and audio-visual fusion. Selected publications of my work can be found below. Representative papers are highlighted.

In this work, we introduced recursive fusion of joint cross-attention across audio, visual and text modalities for multimodal dimensional emotion recognition. We participated in the valence-arousal challenge of 6th ABAW competition and achieved second place.

In this work, we addressed the problem of weak complementary relationships across audio and visual modalities due to sarcastic or conflicting emotions. We address the limitations of cross-attention in handling weak complementary relationships by introducing a novel framework of Dynamic Cross-Attention.

In this work, we addressed the problem of weak complementary relationships for effective audio-visual fusion for person verification using dynamic cross-attention.

Proposed a recursive joint cross-attention model for effective audio-visual fusion for person verification using recursive attention across audio and visual modalities in the videos.

Proposed a recursive joint cross-attention model for effective audio-visual fusion for speaker verification using recursive attention across audio and visual modalities in the videos.

Proposed a recursive joint cross-attention model for effective fusion of audio and visual modalities by focusing on leveraging the intra-modal relationships using LSTMs and inter-modal relationships using recursive attention across audio and visual modalities in the video.

Investigated the prospect of leveraging both intra and inter-modal relationships using joint cross-attentional audio-visual fusion. The robustness of the proposed model is further validated for missing audio modality along with interpretability analysis.

Proposed a joint cross-attention model for effective fusion of audio and visual modalities by focusing on leveraging the intra and inter-modal relationships across audio and visual modalities in the video.

Proposed a cross-attentional model to leverage the intermodal characteristics across audio and visual modalities for effective audio-visual fusion.

Proposed a Holistic Guidance (HG) method that relies on holistic (or non-occluded) data and its distribution in dissimilarity space to train on occluded datasets without the need of any external source.

Proposed a deep learning model for weakly-supervised Domain Adaptation with ordinal regression using coarse sequence level labels of videos. In particular, we have enforced ordinal relationship in the proposed model using gaussian distribution.

In this paper, we have presented a comprehensive taxonomy of weakly supervised learning models for facial behavior analysis along with its challenges and potential research directions.

In this paper, we have proposed a novel framework of weakly supervised domain adaptation (WSDA) with limited sequence-level labels for pain localization in videos.

In this paper, we have proposed a simple yet robust novel approach for the segmentation of high-density crowd flows based on super-pixels in H.264 compressed videos.

Source taken from here