Indirect monitoring of cared person by onomatopoeic text of environmental sound and user’s physical state

Abstract

In this paper, we propose a nonverbal, descriptive method for creating daily life logs, in text format, on behalf of people who require monitoring and/or assistance in taking care of themselves. The users environmental situations are converted into and recorded as onomatopoeic texts in order to preserve their privacy. The users ambient context is detected by the accelerometer, gyro sensor, and microphone in her/his smart device. We propose a soft monitoring system that utilizes nonverbal expressions of both onomatopoeic text logs and symbolic sound expressions that is named Soundgram. We have investigated impressions regarding the monitoring of the elderly and the proposed system via a questionnaire distributed to two groups of potential users, the elderly and middle-aged people, which captures the viewpoint of both the recipient and the caregiver.