A touch-based multimodal and cryptographic bio-human-machine interface

Abstract

The awareness of individuals' biological status is critical for creating interactive and adaptive environments that can actively assist the users to achieve optimal outcomes. Accordingly, specialized human-machine interfaces-equipped with bioperception and interpretation capabilities-are required. To this end, we devised a multimodal cryptographic bio-human-machine interface (CB-HMI), which seamlessly translates the user's touch-based entries into encrypted biochemical, biophysical, and biometric indices. As its central component, the CB-HMI features thin hydrogel-coated chemical sensors and inference algorithms to noninvasively and inconspicuously acquire biochemical indices such as circulating molecules that partition onto the skin (here, ethanol and acetaminophen). Additionally, the CB-HMI hosts physical sensors and associated algorithms to simultaneously acquire the user's heart rate, blood oxygen level, and fingerprint minutiae pattern. Supported by human subject studies, we demonstrated the CB-HMI's capability in terms of acquiring physiologically relevant readouts of target bioindices, as well as user-identifying and biometrically encrypting/decrypting these indices in situ (leveraging the fingerprint feature). By upgrading the common surrounding objects with the CB-HMI, we created interactive solutions for driving safety and medication use. Specifically, we demonstrated a vehicle-activation system and a medication-dispensing system, where the integrated CB-HMI uniquely enabled user bioauthentication (on the basis of the user's biological state and identity) prior to rendering the intended services. Harnessing the levels of bioperception achieved by the CB-HMI and other intelligent HMIs, we can equip our surroundings with a comprehensive and deep awareness of individuals' psychophysiological state and needs.