MAScreen: Augmenting Speech with Visual Cues of Lip Motions, Facial Expressions, and Text Using a Wearable Display

Abstract

Personal protective equipment, particularly face masks, have become increasingly common with the rise of global health issues, such as fine-dust storms and pandemics. Face masks, however, also degrade speech intelligibility by effectively occluding visual cues, such as lip motions and facial expressions. In this paper, we propose MAScreen, a wearable LED display in the shape of a mask, which is capable of sensing lip motion and speech and provides real-time visual feedback of the mouth behind the mask.