Schizosymphony

Schizosymphony merges science, technology, and art to provide a novel perspective on understanding and representing schizophrenia, a complex mental disorder. The project translates functional magnetic resonance imaging (fMRI) data from schizophrenia patients into spatial sound using two spatial sonification techniques: the tonal and the narrative soundscape approach. By transforming brain data into auditory experiences, Schizosymphony aims to bridge the gap between the public and scientific data, fostering empathy and a deeper understanding of schizophrenia.

 

Schizosymphony integrates urban soundscapes to create a cohesive auditory experience, representing moments from everyday life. It assigns unique sound elements to 53 Intrinsic Connectivity Networks (ICNs), which are grouped into 7 Functional Domains (FDs): Subcortical (SC), Auditory (AU), Sensorimotor (SM), Visual (VI), Cognitive Control (CC), Default Mode (DM), and Cerebellar (CB). This process weaves these auditory elements into a unified soundscape, simulating the sensory and mental states often associated with schizophrenia. The resulting auditory experience offers listeners an immersive opportunity to understand the sensory world of individuals with this condition.

 

The fMRI data for this project was sourced from the Function Biomedical Informatics Research Network (FBIRN) dataset and processed through the NeuroMark pipeline. Using spatially constrained independent component analysis (ICA), subject-specific functional components and their corresponding time courses were extracted. These components served as the foundation for designing and spatializing the sonification.

 

Schizosymphony not only raises awareness about mental health through immersive art but also demonstrates the potential of auditory representations of data for practical applications. It offers possibilities for experts to analyze brain imaging patterns audibly, while also exploring innovative therapeutic approaches like music therapy. By blending science, art, and technology, Schizosymphony redefines the way we understand schizophrenia, providing new insights and fostering greater empathy and innovation in mental health advocacy and neuroscience research.

 

As the lead role in the project, I initiated and developed the concept and methods for aesthetic data sonification, proposing a new framework based on fMRI data. Additionally, I created a sound demo and sound installation, utilizing grouped spatialization and data-based mapping to emphasize the significance and flow of the data.