return ✕︎

Immersive Shared Reality

By E. Glen Weyl, Audrey Tang and ⿻ Community

Immersive Shared Reality

Immersive shared reality technologies unlocks a new chapter in human interaction, leveraging cutting-edge virtual reality (VR), augmented reality (AR) and mixed reality (MR) systems. Unlike the deeply personal and sensorially rich exchanges of post-symbolic communication, shared immersive reality presents a broader canvas for human interaction, enabling people to engage in shared, multisensory experiences. This chapter delves into the landscape of immersive technologies, today’s applications, tomorrow’s potential, and the frontier. It shows how immersive technologies may facilitate shared experiences that blend physical and virtual reality, complementing and expanding human experience with interactions that surpass physical, spatial and social limitations. Shared immersive reality creates spaces where communities may converge for socialization, gaming, entertainment, and more, facilitating connections that, while less intense than symbolic communication, are meaningful and emotionally resonant. From virtual reality gatherings that unite people across the globe, to mass online gaming and virtual music festivals, these digital arenas extend the possibility space of shared human experience.

On the horizon, shared immersive reality is poised for rapid expansion. Technological advancements will deepen the sensory integration of these virtual experiences, extending beyond sight and sound to include touch, smell, and even taste. This future, teeming with hybrid reality environments and emotional connectivity heralds a new era of human interaction, where digital spaces not only simulate reality but enhance it, bridging divides and fostering broader understanding. However, immersive reality also has its perils. From the widening surveillance to virtual escapism, these challenges demand thoughtful consideration to ensure that our digital futures augment, rather than diminish, the richness of human experience.

TODAY

Immersive Shared Reality refers to technology that creates a shared virtual environment where users can interact in real-time. This type of “reality” can be considered a subset application of Mediated Reality, a broader term that encompasses various technologies that mediate our perception of reality, including Virtual, Augmented and Mixed Reality (a.k.a. VR, AR, MR).

Venn diagram with four overlapping circles labeled "Virtual Reality", "Augmented Reality", "Mixed Reality," and "Modulated Reality." A larger circle encompasses these, labeled "Mediated Reality."

**Figure 1: Mediated Reality Framework adapted from Mann and Nnlf ’94 **

Shared immersive reality can apply to many human interactions, creating digital spaces where people can connect in meaningful, albeit less intensive and sensory-rich ways than post-symbolic communication. Some of the most common applications are socialization, gaming, entertainment, sports and fitness: Virtual Reality Gatherings: These digital spaces enable people from around the world to interact in a shared virtual environment. Here, avatars represent participants, allowing for expressive movements and interactions that go beyond verbal communication. These virtual gatherings can range from collaborative work meetings to social events, where the sense of presence is amplified by the immersive, 3D environment. Participants experience a sense of togetherness and community, facilitating connections that, while not as intense as physical interactions, are still meaningful and emotionally resonant.

  • Mass Online Gaming: Online multiplayer games create expansive worlds where players collaborate, compete, and strategize together. Communication is a blend of in-game gestures, strategic planning, and quick decision-making, often under time pressure. This environment nurtures a form of camaraderie and collective intelligence, as players become attuned to each other's play styles and tactics.
  • Religious Online Services: In the digital era, religious gatherings have expanded into online platforms, allowing congregations to participate in services and rituals remotely. This form of communal worship, while lacking the physical closeness of traditional services, still offers a sense of shared belief, uniting participants in a common religious experience.
  • Virtual Music Festivals and Parties: With the advent of streaming technology, music festivals and parties have found a new home in the virtual world. Artists like Billie Eilish have already held virtual concerts. Music festivals like Coachella have embraced the benefits of VR without the constraints of physical venues (e.g., selling out).
  • E-Sports Tournaments: E-sports have gained immense popularity, with spectators and players engaging in highly competitive gaming at a professional level. These events, often streamed to vast audiences, create a shared sense of excitement and allegiance among fans.
  • Remote Fitness Classes: The rise of online fitness has brought people together in pursuit of health and wellness. Participants engage in synchronized workouts, yoga sessions, or dance classes from their own homes, sharing a common goal and a sense of group motivation.
  • Virtual tourism: travelers can experience remote places, walking through historic cities or visiting foreign landscapes from the comfort of the homes. This technology enables travelers to virtually walk through historic cities, marvel at natural wonders, and immerse themselves in foreign landscapes.

Each of these examples showcases the different ways in which technology is facilitating new forms of shared immersive reality. These experiences, while less intense than the sensory-rich interactions of post-symbolic communication, still expand the possibility space of expression, understanding, and impact, pushing the boundaries of how we communicate, foster connections, build communities, and creating shared experiences among groups of people.

TOMORROW

We envision a near future where advancements in technology not only enhance these experiences in immersive shared reality but also introduce novel ways for people to connect, learn, and empathize with one another on an unprecedented scale. The future of shared immersive reality promises to making distant or imagined experiences palpably real, enveloping users in a synthetic world that simulates multiple senses simultaneously. While sight and sound have been the traditional focus, new sensors and actuators promise to deepen integration of touch, smell, and even taste. Haptic feedback systems will replicate the subtleties of physical contact. Olfactory technology will enable fragrances and odors to be part of storytelling, education, and even retail experiences in VR. Taste retargeting [1] will unlock virtual dining experiences by altering taste perception delivering chemical modulators to the mouth. Here are a few envisioned advancements and novel examples that extend the concept of shared immersive reality into new dimensions: Hybrid Reality Environments: Leveraging augmented reality (AR) and virtual reality (VR) in tandem, these environments blend physical and digital elements seamlessly. Imagine attending a conference where remote participants appear as full-size holograms, capable of interacting with physical objects and participants in real-time. This blurs the line between who is present physically and who is digital.

  • Enhanced Emotional Connectivity in Social VR: Emerging technologies aim to transmit nuanced human emotions and physical sensations through VR, using advanced haptic feedback, biometric sensors, and emotional AI. This could enable users to feel the warmth of a handshake, the pressure of a hug, or even the subtleties of emotional expression conveyed through a virtual avatar, deepening connections and empathy between participants and enabling those with visual or auditory impairments to engage through other senses.
  • Massive Multi-user Online Laboratories (MMOLs): Scientists can collaboratively conduct experiments in a shared virtual laboratory. MMOLs could facilitate real-time collaboration on scientific research and education across the globe, breaking down barriers to access and enabling a form of immersive, collective discovery.
  • Virtual Reality Civic Spaces: Digital replicas of civic centers, town halls, and community spaces where people can gather to discuss, debate, and make decisions about their communities. These spaces would allow for a more inclusive and accessible form of civic engagement, enabling participants to engage in local governance or community planning processes from anywhere in the world.
  • **Immersive Educational Experiences: From virtual field trips to interactive historical reenactments, educational content will become more immersive, allowing students of all ages to explore and learn in ways that are engaging, memorable, and more impactful than traditional methods. Imagine history classes where students can witness historical events unfold or science classes where they can walk through a functioning human heart.
  • Cross-Cultural Exchange Platforms: Platforms specifically designed to foster understanding and empathy between diverse cultural groups by immersing users in the experiences of people from different backgrounds. Through narratives, rituals, and daily life activities, these platforms could use VR and AR to bridge cultural divides and build a global sense of community. For example, language learning applications use these to immerse users in the linguistic and cultural background of others. Another example is the the Portals Policing Project [2], which shares the lived experiences of people with law enforcement in a controlled, yet realistic virtual chamber, improving understanding and trust on both sides.
  • Environmental and Climate Change Simulations: Interactive simulations that allow users to experience the potential impacts of climate change firsthand. For example, the Tree Project [3] demonstrates how immersive VR can evoke empathy and compassion for the natural environment by transforming the user into a rainforest tree and exposing them to the threats of deforestation and climate change.
  • Virtual Reality Therapy Sessions for Mental Health: Leveraging the power of VR to create therapeutic environments, sessions could offer healing and comfort to individuals facing mental health challenges. Participants could be transported to calming natural settings or engage in guided meditations designed to alleviate stress, anxiety, and depression. Tailored experiences could simulate scenarios for those with PTSD to safely confront and work through their traumas under the guidance of a therapist. Similarly, social anxiety could be addressed through simulated social interactions, allowing for practice and exposure in a controlled, supportive environment. This application of VR not only broadens access to mental health services but also introduces a novel, highly personalized approach to therapy.

As these technologies mature, they are increasingly harnessed to not just simulate reality but to enhance it, creating a bridge between diverse cultures and fostering a global community of shared experiences and mutual understanding regardless of one’s origin or language. These envisioned applications of shared immersive reality hold the potential to transform how we interact with the world and each other, fostering understanding, empathy, and collaboration across all facets of human endeavor.

Frontiers of Immersive Shared Reality

As we gaze to the horizon of immersive shared reality, the very nature communal experience and human connection undergoes a profound metamorphosis. Imagine stepping into a world where shared virtual spaces are not mere simulations, but extensions of our physical reality, offering experiences that are as rich and complex as those encountered in the tangible world. In this future, immersive shared reality technologies enable a fusion of senses, thoughts, and emotions. At the frontiers of Immersive Shared Reality, we are not merely spectators but active participants in a revolution of multisensory integration [4]. Imagined Worlds and Dream Sharing: more sophisticated and controlled use of sensory inputs (e.g, smell, taste, visual and auditory), will enable participants to generate and share realities that deeply resonates with participants’ emotions and memories. Such stimuli, when reactivated during sleep, not only can enhance these memories [5], but facilitate sharing altered states of consciousness [6] and shared lucid dreams [7]. Participants will be able to explore the subconscious playground of the human mind together. Simulated Worlds: Virtual environments can simulate realities—both the future and past—under different conditions. For example, participants will be able to experiment with scenarios of climate change, such as rising sea levels or the impact of extreme weather events, making distant concepts an immediate and personal experience. With affective computing, the system may adapt the environment based on the user’s response, physiology as well as memories or preferences, creating a feedback loop that heightens awareness and empathy.

  • Virtual Design Studios: community members, architects, and engineers may come together to co-create the green spaces of tomorrow to redefine “planning.” Participants virtually touch the bark of trees slated for planting and inhale the fragrant blossoms intended for the gardens. Participant feedback can modify the simulation in real-time, enabling sensorial immersion into different visions for a project.
  • Collective Memory Palaces: Envision virtual environments where entire communities can deposit, share, and experience collective memories and knowledge. These memory palaces serve not only as repositories of communal wisdom but as spaces where individuals can relive historical events or explore the collective psyche of humanity, fostering a deeper understanding and connection across generations.
  • Empathy Amplifiers: Advanced immersive technologies could allow us to experience the world through the eyes of another. This direct sharing of experiences would serve as an empathy amplifier, dissolving prejudices and fostering a profound sense of unity and understanding among diverse groups of people. Envision simulations that allow individuals to live through the collective experiences of entire communities, nations, or civilizations, feeling their struggles, joys, and challenges as their own. This could serve as a powerful tool for education and conflict resolution, promoting peace on a global scale.
  • Global Consciousness Networks: Imagine a future where people can connect their consciousness to a global network, sharing thoughts, emotions, and experiences in a dynamic, evolving stream of collective awareness. This network would enable a form of communication and connection that goes beyond language, allowing for an unparalleled synchronization of human intention and action towards global challenges.
  • Inter-species Communication Platforms: Beyond human-to-human interaction, immersive shared reality could extend the boundaries of communication to include other species. By translating non-human languages and experiences into formats we can understand and vice versa, these platforms could foster an unprecedented level of empathy and cooperation between humans and other life forms on our planet.
  • Digital Legacy Realms: Future immersive technologies could allow individuals to create digital legacies—entire worlds crafted from their memories, thoughts, and experiences. These realms would not only serve as a form of immortality but also as a means for future generations to explore the lives and insights of their ancestors in a deeply personal and interactive way. -Collective Creativity Spaces: These digital platforms would enable artists, musicians, writers, and creators of all kinds to collaborate in real-time, across the globe, in shared virtual spaces. Here, ideas and inspirations merge in a communal creative flow, leading to art and innovation that truly represents the collective human spirit, transcending individual capabilities.

As we embark on this journey towards a future of immersive shared reality, we stand on the brink of redefining human experience and collaboration. The technologies that lie ahead promise not just advancements in the way we interact with the world, but a revolution in the way we perceive, connect and innovate. In this new era, the barriers between individual consciousness and collective experience become more fluid, heralding a future where our shared realities foster a deeper unity and yet more creative collaborations.

LIMITS

Unlike the intimate, direct exchange of thoughts and emotions envisioned in post-symbolic communication, shared immersive realities unlocks new dimensions for human interaction and coordination from simple social interaction to education, work, and entertainment— bringing with them a distinct set of limitations and ethical concerns. If the Matrix is a dystopia of post-symbolic communication, a similar and a fitting dystopian parallel can be drawn from “Ready Player One” by Ernest Cline, which was later adapted into a film directed by Steven Spielberg. In “Ready Player One,” the Earth's environmental decay and socioeconomic disparities have driven the majority of the population to seek refuge and fulfillment in the OASIS (Ontologically Anthropocentric Sensory Immersive Simulation), a vast virtual reality universe that offers an endless array of worlds and experiences, far surpassing the bleak prospects and the desolation of the physical world. OASIS epitomizes several critical dystopian themes:

  • Virtual Escapism: The allure of the OASIS leads to widespread neglect of the physical world, as individuals prefer the comfort and excitement of virtual experiences to addressing real-world problems. The dependency on the OASIS for fulfillment and the consequent disengagement from the physical world portray a dystopian outcome where society becomes addicted to virtual experiences, leading to widespread neglect of personal health, relationships, and civic responsibility.
  • Diminished Physical health: Immersing oneself in alternative realities for extended periods can lead to psychological effects, such as difficulty distinguishing between virtual and physical experiences or feeling disconnected from real-world social bonds. The ready availability of an idealized digital escape could impact mental health, leading to isolation or a diminished ability to cope with real-world challenges.
  • Digital Divide: A new digital frontier risks widening the gap between those with access to the latest technologies and those without. As these immersive experiences become more integral to social and professional life, lack of access could marginalize individuals and communities, Within OASIS, wealth and access to digital resources greatly influence one's experiences and opportunities, mirroring and amplifying real-world socioeconomic disparities.
  • Physical Health Implications: Prolonged engagement in virtual environments raises concerns about physical health, including the effects of extended screen time on vision, and the sedentary lifestyle associated with immersive digital activities. Balancing the allure of virtual worlds with the need for physical activity and real-world interaction becomes a crucial health consideration.
  • Corporate Control, Surveillance, and Monopolization: Shared immersive realities blur the lines between public and private, where digital spaces can be simultaneously intimate and open to wide audiences, or observed by corporate service providers. The battle for control of the OASIS highlights the dangers of a single entity owning and operating the digital realm, raising concerns about monopolization, surveillance, and the commodification of personal data and experiences in shared immersive realities.
  • Identity and Authenticity: The freedom to create and adopt any persona in the OASIS brings into question the concepts of identity and authenticity. It illustrates the potential for anonymity and fluid identity in shared immersive realities to complicate trust and relationships, as well as the possibility of losing one's sense of self.

"Ready Player One" serves as a cautionary tale for the development and adoption of shared immersive reality technologies, emphasizing the need for balance between digital and physical lives, and the preservation of individual privacy and autonomy. Addressing these concerns requires a careful integration of participatory governance systems, like democratic voting mechanisms, into virtual environments, as well as market interactions in “lesser bandwidth” channels that keep participants engaged in the physical realm. Grounding participants and striking a balance between digital engagement and their real-world roles and responsibilities is key to harnessing the strength of immersive technologies, without succumbing to their weaknesses.


  1. Jas Brooks, Noor Amin, and Pedro Lopes. 2023. Taste Retargeting via Chemical Taste Modulators. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST '23). Association for Computing Machinery, New York, NY, USA, Article 106, 1–15. https://doi.org/10.1145/3586183.3606818 ↩︎

  2. https://www.justicehappenshere.yale.edu/projects/portals-policing-project ↩︎

  3. www.treeofficial.com ↩︎

  4. Cornelio, Patricia, Carlos Velasco, and Marianna Obrist. "Multisensory integration as per technological advances: A review." Frontiers in Neuroscience (2021): 614. ↩︎

  5. Amores Fernandez, Judith, et al. "Olfactory Wearables for Mobile Targeted Memory Reactivation." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 2023. ↩︎

  6. Carr, Michelle, et al. "Dream engineering: Simulating worlds through sensory stimulation." Consciousness and cognition 83 (2020): 102955. ↩︎

  7. Konkoly et al. Real-time dialogue between experimenters and dreamers during REM sleep. Current Biology, 2021 DOI: 10.1016/j.cub.2021.01.026 ↩︎