AffectPro: Towards Constructing Affective Profile Combining Smartphone Typing Interaction and Emotion Self-reporting Pattern

Abstract

The ubiquity of smartphones and the widespread usage of text entry by soft keyboard in different instant messaging applications (e.g., WhatsApp, FB messenger) have opened the possibilities of inferring emotions from longitudinal typing data. To build this emotion inference engine, we apply machine learning models on features extracted from user's typing patterns (not content). However, one major challenge encountered while developing the emotion inference model is the requirement of individual training data as typing patterns are often person-specific. In this paper, we investigate the possibility of combining typing pattern with emotion self-reporting to identify a group of similar users so that the training data among these users can be shared to fulfill the requirement of personalized dataset. We develop a framework AffectPro, which quantifies the typing interaction behavior (e.g., typing speed, error rate) and self-reporting pattern (e.g., emotion state transition probability) to construct the affective profiles of users. We evaluated AffectPro in a 6-week in-the-wild study involving 28 users, who used an Android application encompassing a custom keyboard to perform all their typing activities, and to report their instantaneous emotions. We extracted different typing signatures and self-report behavior details from the collected dataset (≈ 5000 typing sessions, ≈ 108 hours of typing data) to construct the affective profile of users. Our results demonstrate similarity across users in terms of typing signature, emotion self-reporting pattern, and a combination of both; which can be leveraged to share training data among similar users to overcome the challenges of personalized data collection.