Fundamentals of All the Realities: Virtual, Augmented, Mediated, Multimediated, and Beyond

Abstract

This chapter introduces the different “realities”: virtual, augmented, (deliberately) diminished, mixed, mediated, multimediated, and phenomenological. The contributions of this chapter include (1) a taxonomy, framework, conceptualization, etc., of all of the “realities”; (2) a new kind of “reality” that comes from nature itself, which expands our notion beyond synthetic realities to include also phenomenological realities; and (3) methods of using phenomenological reality as a means of visualizing as well as understanding hidden phenomena present in the world around us. VR (virtual reality) replaces the real world with a simulated experience (a “virtual” world). AR (augmented reality) allows the real world to be experienced while at the same time, adding to it, a virtual world. Mixed reality provides blends that interpolate between real and virtual worlds in various proportions, along a “virtuality” axis, and extrapolate to an “X-axis” defined by “X-Reality” (eXtended reality). Mediated reality goes a step further by mixing/blending and also modifying reality, including, for example, deliberate diminishing of reality (e.g., a computerized welding helmet that darkens bright subject matter while lightening dark subject matter). This modifying of reality introduces a second axis called “mediality.” Mediated reality is useful as a seeing aid (e.g., modifying reality to make it easier to understand) and for psychology experiments like George Stratton’s 1896 upside-down eyeglasses experiment. Multimediated reality (“all reality” or “All R”) is a multidimensional multisensory mediated reality that includes not just interactive multimedia-based “reality” for our five senses but also includes additional senses (like sensory sonar, sensory radar, etc.), as well as our human actions/actuators. These extra senses are mapped to our human senses using synthetic synesthesia. This allows us to directly experience real (but otherwise invisible) phenomena, such as wave propagation and wave interference patterns, so that we can see radio waves and sound waves and how they interact with objects and each other, i.e., phenomenological reality. Moreover, multimediated reality considers not just multiple axes in addition to the X-Reality axis but also that the origin of the axes exists at zero sensory stimuli. In this way, we can account for various (virtual, augmented, etc.) realities in a sensory deprivation float tank. Consider, for example, wearing an underwater VR headset while floating in a sensory deprivation tank. Then consider an augmented reality headset while floating in an immersive multimedia pool. Consider Internet-connected immersive multimedia water therapy and music therapy pools that allow multiple users to play a hydraulophone (underwater pipe organ) while meditating together in a hydraulically multimediated collective. These are examples of what we call “fluid (user) interfaces,” and they fall outside the range of any previously existing “reality taxonomy” or conceptualization.