Preface

Abstract

Humans and the infocommunications networks surrounding them are becoming entangled at various levels, ranging from the level of bioelectronic interfaces, through the level of multisensory interactions with personal devices, to the higher level at which the automated sensing, representation and understanding of (collective) human behaviors becomes possible. Through this process of entanglement, humans and infocommunications will soon coexist as an entangled web, which can be expected to lead to an augmentation and merging of both natural and artificial cognitive capabilities. The resulting synergic combinations of humans, devices, infrastructure and environment—identifiable from the perspective of higher-level cognitive capabilities—can be seen as newly emerging cognitive entities. The goal of the field of cognitive infocommunications (CogInfoCom) is to investigate the ways in which cognitive entities are created as a result of human-ICT entanglement, and how new cognitive capabilities are synthesized. The field adopts a multi- and interdisciplinary approach in achieving this goal. Thus, besides aiming to create a common language, it is also the goal of CogInfoCom to establish a common way of thinking about relevant problems, and a common approach towards designing new technologies. This is challenging, given the differences between analysis and synthesis-oriented viewpoints reflected in the engineering and cognitive sciences. This special issue, entitled “Multimodal Biases in CogInfoCom Networks”, presents a collection of papers broadly focused on the influence of cognitive biases on cognitive capabilities. It is well known that human decisions and actions are often mediated through strong biases. The effects of such biases are compounded when multiple modalities (e.g., vision, speech, gestures, emotional expressions, etc.) are used in conjunction. As a result, engineering systems that work together and co-evolve with humans are expected to take such factors into consideration. This can be done in at least two ways: either the perceptual parameters of (multi-)sensory feedback (e.g., the level of urgency conveyed by warning signals, the quality of voice used in speech-based feedback, the brightness/contrast of visual feedback, etc.) is modulated, or the feedback information is left un altered, but instead the control signals (i.e., the effects of the user’s actions) are modulated so as to counter-balance the potential effects of cognitive biases. Papers range from the more theoretical to those which introduce practical applications. Although the contributions of these papers are often multi-layered and multi-faceted, in the following we make an attempt to briefly describe their key points and the relationships between them in terms of their scope and results. At the level of interaction with personal devices, several papers focus on multimodal aspects of assistive technologies. Balata and his colleagues introduce a mobile graphical user interface for elderly users, called Koala, that is based on a set of qualitative and quantitative studies conducted by the authors. Csapó et al. provide a survey of mobile technologies relying on auditory and haptic feedback, and give an up-to-date outline of challenges in mobile interaction design with respect to users with special needs. On a related subject, Sövény et al. present a solution called Blind Guide, which is a pedestrian navigation system that uses audio feedback to support blind users in their everyday lives. The paper by Posusta et al. describes an investigation on various bi-manual control schemes in myoelectric interfaces, also targeted at users with cognitive impairment. Two further papers of the special issue can be seen as dealing with the level of interaction between users and personal devices. First, the contribution made by Kundra et al., proposes an improvement to the orientation sensing capabilities of wearable technologies based on a probabilistic scheme for optical flow analysis. Second, the paper by Török et al. investigates biases in the ventriloquism effect (which, as a phenomenon in which the perception of audio direction is affected by visual stimuli, is a bias in itself). The paper shows that the direction of visual stimuli displacment (horizontal as opposed to vertical), as well as the orientation of the auditory stimuli both influnce the strength of the ventriloquism effect. At a higher level of (social) behavior and cognition, focusing in particular on technology-mediated healthcare, Almog et al. show that different forms of technological mediation in telemedicine lead to different ways in which distress can be relieved (i.e. in terms of receiving attention, being understood, being present and mutual emotional connectedness). In another paper loosely related to this topic, Mottura et al. present a virtual reality system called REAPP, which aims to strengthen aspects of awareness and participatory experience in the rehabilitation of post-stroke patients. Focusing on a similarly high level of behavior, Bruijnes et al. provide a detailed presentation of how the interpretation of interpersonal stances in police interrogation scenes, in terms of e.g. friendliness and dominance, are influenced by both subjective parameters and professionality level when the scenes are performed by actors with various degrees of proficiency. Finally, the paper by Lengyel et al. presents important ideas on how high-level interface design can be facilitated by formal (graph-based) model transformations. It is worth mentioning contributions made in the special issue to the areas of speechability and socio-cognitive ICT separately, as the formulation of these areas is a relatively new development in the field of CogInfoCom. Briefly stated, speechability aims to reduce the fragmentation in research on speech technologies by attempting to link cognitive linguistics with verbal and non-verbal social communicative signals; while socio-cognitive ICT focuses on ways in which collective (social) behaviors are used either to inform and support individual user interactions, or the prediction/analysis of collective behaviors and events. Related to the area of speechability, the paper by Raptis et al. focuses on the analysis and synthesis of emotional speech, based on hierarchical clusters of emotional groups extracted both from text and speech, without recourse to any a priori emotional model. The results of the paper demonstrate well how the situated aspects of speech can be incorporated into generative models to enhance human-ICT capabilities. The last two papers of the special issue can be seen as relevant to the area of socio-cognitive ICT. Szegletes et al. present new ideas for socio-cognitive gamification, in which (collective) social-emotional signals are used to dynamically modify the difficulty parameters of educational games. The paper presents one possible implementation of the framework proposed by the authors. Finally, the contribution by Nakai et al. describes a set of experimental evaluation tools that are motivated by Kansei engineering, and can be used to obtain holistic assessments both in subjective and objective dimensions. The concepts presented in the paper are demonstrated through the evaluation of a serious game. In summary, the papers collected in this special issue cover a wide range of topics which contribute to a deeper appreciation of the many ways in which modern ICT facilitates the merging of human and artificial cognitive capabilities, and the ways in which human (and increasingly, human-ICT) cognitive biases fit into this picture.