An Exploratory Qualitative Investigation: Multi-modal AI Technology Concepts for Non-verbal Feedback During Remote Work Meetings

Abstract

Prior research has shown that computer-mediated communication introduces barriers to effective non-verbal communication. This creates potential challenges as workers increasingly collaborate remotely in a post-pandemic world. To further investigate these barriers, we conducted an exploratory study with five remote workers featuring qualitative user testing of concepts designed to facilitate non-verbal feedback using multi-modal AI technologies. Prior to the study, a team of UX designers created low-fidelity concept prototypes using different variables important for participant feedback (that is, modality, type, user control, visibility, granularity, timing, and categories). Using these prototypes, we conducted semi-structured interviews (90-min long) with scenario-based questions. The results showed that the participants perceived the prototypes for multi-modal feedback positively, especially for the multi-tasking scenarios. However, this perception differed based on different roles: Presenters versus listeners. In general, the participants indicated that the presenters would be more interested in feedback than the listeners, as long as that feedback was uncluttered and actionable. By contrast, when commenting as listeners (that is, as individuals whose feedback would be monitored), the participants expressed concerns over the accuracy of AI models, a sense of being monitored (privacy concerns), and potential social risks to provide uncomfortable feedback. As a future work, we plan to iterate on the current prototypes to address these concerns, and deploy a larger study with refined and more functional prototypes.