return ✕︎

Post-Symbolic Communication

By E. Glen Weyl, Audrey Tang and ⿻ Community

Post-Symbolic Communication

Overlooking Tokyo, nestled within the National Museum of Emerging Science and Innovation (the Miraikan) lies the Park of Aging, a realm where time bends into the distant future, offering a rare portal to your mind and body after years worn by aging. Video games blur vision, mimicking cataracts. Sounds are stripped of high pitches. In a photo booth that mirrors the trials of aged perception, facial expressions fade and blur. The simple act of recalling a shopping list committed to memory becomes an odyssey, ceaselessly interrupted in a bustling market. Walking in place on pedals with ankle weights, and leaning on a cart simulates the wear of time on the body, and the weight of age on posture. The Park of Aging is not just an exhibit but an immersive conversation across time, a dialogue with your older self through senses—sight, sound, and the aches and pains of age. This journey of empathy extends beyond the future self, to also foster a deeper connection with a present overlooked cohort: the elderly.

Private

The Park of Aging is a poignant example of proprioceptive, post-symbolic communication, where participants receive information through an intimate, sensorial experience beyond merely interpreting words and symbols, utilizing all sensemaking of the body; the Park immerses participants in the sensations of being old, unlocking the first-hand experience of the deteriorating senses, including seeing and hearing words and symbols. While the Park is a novel, temporal conversation with aging, experiences of proprioceptive, non-symbolic communication today are ubiqutious and include mediation, psychedelics, religious experience, romantic intimacy, dance, yoga, combat, and sports. Not coincidentally, these experiences which harness information gathered from many senses (“higher-bandwidth communication”) also are correlated with long-lasting human bonding and connection when shared. Perhaps it is the diversity of information simultaneously presented to the senses (depth) that contribute to the significance of the experience and therefore strength of connection and “union.”

Technological innovations such as neural-interfaces, mediated reality, and generative AI expands the possibility space of post-symbolic communication, where unions within oneself and among people can occur not simply in person, but across temporal, physical, and social distance facilitated by technology. In this chapter, we describe these technologies and explore the frontiers, examining how these technologies could revolutionize interpersonal connections, education, and collaboration, enabling the transmission of thoughts, feelings, and sensory experiences beyond the compression of words and symbols. We consider the opportunities to rectify cultural misunderstandings and conflicts, by fostering profound empathy and shared experience. Yet, where ideas flow as seamlessly as emotions, we also explore the risks of connectivity, including surveillance, homogenization, disconnection and oblivion.

TODAY

Post-symbolic communication, a term coined by Jaron Lanier [1], ventures beyond the realm of language and symbols to explore the potential for direct and immersive shared experience harnessing all senses. Our first experience of non-verbal communication is in the natal womb; the synchronization of heartbeats between a mother and her unborn child, especially when the mother breathes rhythmically, suggests an intrinsic communication pathway [2]. As we develop, humans convey information nonverbally through body language, facial expression, tone, touch, laughter, crying, and even smell. Biochemical messengers can convey emotional states and trigger responses in others, often unconsciously. For instance, research has shown that human sweat contains compounds that, when detected by others, can convey stress or fear, influencing the receiver's perception and behavior. [3]. We also see glimpses of this post-symbolic potential in shared, intensive experiences among humans:

Dance: Stepping deeper into music, two participants synchronizing their movements both feel their body and anticipating their partner’s movements to create shared experience. Traditional dances like the Adumu ritual of the Maasai people also serve as a way for communal shared experiences through synchronicity.

Combat: on the battlefield, soldiers experience heightened senses, adrenaline, and a battery of sounds, leading to intense awareness of their surroundings and their comrades. Non-verbal cues become critical for survival and strategy. This shared high-stress environment creates a bond and deep understanding and trust develop among combatants.

Sports: in team sports, players develop a keen sense of each other's presence and movements. Teammates often anticipate each other's actions and work as a cohesive unit, relying on signals communicated through sound, movement and hand gestures. This synchrony is somewhere between dance and combat, also fueled by the shared goal of winning.

Romantic Intimacy: through touch, eye contact, and emotional vulnerability, partners create a unique, shared experience. This connection is built on mutual trust and understanding, often going beyond words to a deeper form of non-verbal communication.

Religious Experience: in mystical practices like Sufism, participants collectively engage in rituals like whirling and chanting. These shared spiritual practices engage the senses in the same way, and create a sense of unity and connection with something greater than oneself, yet nonetheless foster a religious community.

Yoga: in a yoga class, practitioners move together through sequences of poses, guided by the rhythm of their breath. Despite the individual nature of the practice, there is a collective harmony in the movements and the shared goal of health and peace, as well as a heightening nervous system through stretches and poses. This shared physical and meditative experience fosters a sense of communal energy and focus.

Each of these contexts illustrates how shared experiences, beyond the scope of verbal communication, can create deep bonds and understanding between participants through intensive, shared sensorial inputs. Yet, communication can also happen within oneself, as a form of introspection, yielding some of the most profound and transformative experiences.

Meditation: a practice of inward focus and mindfulness, meditation has a diverse set of practices across a range of traditions. One common practice is to observe one's own thoughts and feelings without judgment, fostering a greater recognition and understanding of one’s own mental and emotional states, offering an opportunity to bridge and reconcile internal differences and contradictions.

Psychedelics: The use of psychedelics can lead to profound internal experiences and altered states of consciousness that disrupt usual patterns of thought and perception, opening alternative narratives to sensemake. Users often report experiencing deep introspective insights, a sense of oneness with the universe, confronting deep-seated emotions and memories, or communication with seemingly non-embodied forms of consciousness.

Prayer: Similarly, through prayer, contemplation, or participation in religious rituals, individuals open communication to something other than themselves. Whether through a feeling of divine presence or an internal sense of clarity and peace, these experiences can be deeply impactful and transformative, and foster a greater connection to themselves and their place in the universe.

Communication today is rich with examples that touch upon the edges of post-symbolic communication. From physiological signals to chemosignals we are just beginning to learn the non-verbal, information-dense modes that can synchronize human experience and form the deepest human bonds between oneself and other humans, groups, and the universe broadly.

TOMORROW

Today, we stand on the cusp of a cambrian explosion in technologies that can advance post-symbolic communication to unlock immersive, shared experiences that enable participants to communicate physiological and even psychological phenomenological states of being. The Park of Aging is an example of an early experiment, but we can expect deeper, immersive wholly sensorial experiences with novel integrations of technology that combine in supermodular ways to enable us to communicate ideas and emotions in a more direct and visceral manner for richer, shared experiences. A number of technologies include:

Neural and Brain-Computer Interfaces (BCI): devices that connect directly to the brain capture neural activity and offer a direct pathway for communicating complex thoughts and emotions. BCIs enable direct communication between the brain and external devices. Future developments could allow for the sharing of thoughts, emotions, and experiences directly from one mind to another, allowing for an unprecedented level of interaction.

Haptic Feedback Technologies and homuncular flexibility [4]: Haptic devices provide tactile sensations, simulating touch and physical interactions in virtual environments and allowing users to feel and respond to virtual stimuli as if they were real. Similarly, with homuncular flexibility, individuals can learn to control virtual bodies that differ significantly from their own physical form, transcending the limitations of their physical bodies.

3D Audio and Immersive Soundscapes: Advanced sound technologies that create three-dimensional auditory experiences can deeply enhance the sense of immersion in a virtual space, conveying emotions and atmosphere in a way that traditional stereo sound cannot.

Wearable Tech for Emotion and Physiological State Tracking: Devices that monitor heart rate, skin conductance, and other physiological markers can provide insight into a user's emotional and physical state, enabling shared experiences that are responsive to these states.

Advanced Projection Mapping and Spatial Computing: Technologies allow for the transformation of physical spaces into interactive, digitally augmented environments. They can create shared, multi-sensory experiences that blend the physical and digital worlds.

Neurofeedback and Mindfulness Apps: These applications use real-time displays of brain activity to teach self-regulation of brain function. They can be used for meditation, mental health therapy, and enhancing self-awareness and empathy.

Neuromodulation Neuromodulation techniques, such as deep brain stimulation have implications for enhancing cognitive function, treating neurological disorders, and even augmenting human capabilities. However, they also introduce neuroethical challenges such as moral bio-enhancement(MBE) - the use of biomedical technology to morally improve humans.

With the advent of generative AI, these technologies will furthermore enable us to generate visual representations of our thoughts at a pace that closely mirrors the speed of our imagination, unlocking responsive, adaptive environments or characters in virtual spaces that can react in real-time to users' emotions, actions, or choices beyond simple interpretation of natural language inputs [5]. Researchers have already shown how brain implants can connect the intentions of a paralyzed patient into physical movements [6], demonstrating the remarkable potential of neural interfaces to bridge the gap between thought and action. Combined, these capabilities enable transmission of ideas and emotions that can occur directly and seamlessly, and have profound implications for how we share and understand each other's internal experiences, creative visions, aspirations and even past traumas to facilitate reconciliation, healing and forgiveness. For example, imagine a child immersing themselves in the sensorial experiences of their parents at the same age. Or two cultural groups at war, experiencing the pain and loss of their adversary’s family members. With more inclusive and empathetic ways of interacting, these technologies can unlock communication that is not only post-symbolic but also deeply empathetic, as individuals share experiences and emotions in their most unfiltered, raw and uncompressed form beyond the boundaries of time, physical and even social distance. Such technological advances, when paired with our growing understanding of human perception and cognition, are paving the way for a future where communication encompasses a much broader range of human experiences and states.

FRONTIERS

In the far future, the evolution of non-symbolic communication promises to profoundly reshape our understanding of intimacy and the very essence of life's milestones, such as childhood and relationships. Imagine a world where the boundaries of personal experiences blur, redefining intimacy not as a physical or emotional proximity but as a deep, seamless sharing of consciousness. Telepathy, once a realm of science fiction, becomes an attainable reality, allowing for the direct transmission of thoughts, feelings, and sensory experiences from mind to mind. This profound leap in communication transcends the need for spoken words or physical gestures. The implications of such a capability are monumental. It could lead to an unprecedented level of understanding and empathy between individuals, as barriers erected by language and cultural differences dissolve. Human relationships evolve into deeper, more meaningful connections where misunderstandings are rare, and empathy is abundant. Children, in this new paradigm, grow not just by learning from others' words or observing actions but by immersing themselves in the lived experiences of others, including their ancestors, spanning different cultures and epochs. This experiential osmosis accelerates empathy and wisdom, fostering a society where learning is as much about absorbing direct experiences as it is about traditional education.

Long-distance relationships, too, can expect to undergo a radical transformation. Physical distance becomes a triviality in the face of technologies that allow for the sharing of thoughts, emotions, and sensory experiences in real-time, irrespective of geographic separation. Lovers, friends, and family members can experience each other's joys, sorrows, and mundane moments as if they were in the same room, creating a form of intimacy that transcends physical presence. This paradigm shift brings profound changes in societal structures – the traditional nuclear family could give way to more fluid, globally interconnected familial networks. As we steer towards this horizon, the very fabric of human connection and communication is poised to undergo a metamorphosis, redefining what it means to be intimately connected and be “human.”

LIMITATIONS

The journey into post-symbolic communication is not without its perils. The very tools that promise deeper connection could also lead to a loss of individuality. Worse, participants could be surveilled, leaving our thoughts, emotions, and motivations open to manipulation; any window into our mind is also a window of influence. At the extreme, there is a risk that participants no longer have a private inner world while also disconnecting from the outer world into the “Matrix.” Unchecked visibility into our minds allows for unprecedented levels of manipulation and control, potentially by technology companies or governments that distract humanity with an alternate reality, or simulation. As humans lose touch with the reality of the physical world, over-reliance on telepathic communication could lead to the atrophy of traditional communication skills and cultural practices, with people becoming dependent on direct mental connections. Furthermore, in a world where the boundaries between self and other blur, the sanctity of individual thought and experience could be threatened. High-bandwidth communication could lead to a homogenization of thoughts and experiences as individual perspectives merge into a collective consciousness, erasing our differences.

Balancing telepathic communication with lower-bandwidth, structured forms of communication is essential to preserve privacy, autonomy, diversity, and human governance in the future. Communication, like speech and text, is less direct and immediate than telepathy but is structured and deliberate. It requires the sender to formulate thoughts into words or sentences, providing a level of control and reflection that instantaneous telepathic communication lacks. Markets and voting systems also serve as quintessential examples of lower-bandwidth, structured forms of communication, offering a counterbalance. In markets, the myriad decisions made by consumers and producers are communicated through the price mechanism. This system, while less immediate and detailed than telepathic communication, provides a structured and aggregated way of expressing preferences and values. It allows for privacy in decision-making, as individuals do not have to reveal the full spectrum of their thoughts and motivations. Similarly, voting is a deliberate, structured form of communication where individuals express their political and social preferences at a fixed point in time. Unlike continuous and invasive telepathic streams, voting encapsulates the will of the populace in a manner that is both manageable and interpretable, preserving the autonomy of the individual voter. This structured approach is crucial in maintaining a balance between efficient communication and the safeguarding of personal autonomy, privacy, and democratic processes, acting as a vital check against the overreach of an all-encompassing telepathic matrix.


  1. Lanier, Jaron. You are not a gadget: A manifesto. Vintage, 2011. ↩︎

  2. Van Leeuwen, P., Geue, D., Thiel, M., Cysarz, D., Lange, S., Romano , M. C., Wessel, N., Kurths, J., & Groenemeyer, D. H. (2009). Influence of paced maternal breathing on fetal–maternal heart rate coordination. PNAS, 106(33), 13661-13666. https://doi.org/10.1073/pnas.0901049106 ↩︎

  3. Fernandez, Judith Amores. Olfactory interfaces: toward implicit human-computer interaction across the consciousness continuum. Diss. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2020. ↩︎

  4. Andrea Stevenson Won, Jeremy Bailenson, Jimmy Lee, Jaron Lanier, Homuncular Flexibility in Virtual Reality, Journal of Computer-Mediated Communication, Volume 20, Issue 3, 1 May 2015, Pages 241–259, https://doi.org/10.1111/jcc4.12107 ↩︎

  5. Huang, Han, et al. "Real-time Animation Generation and Control on Rigged Models via Large Language Models." NeurIPS Workshop on ML for Creativity and Design (2023) ↩︎

  6. Lorach, H., Galvez, A., Spagnolo, V. et al. Walking naturally after spinal cord injury using a brain–spine interface. Nature 618, 126–133 (2023). https://doi.org/10.1038/s41586-023-06094-5 ↩︎