Disorders of consciousness (DoC) affect millions of people worldwide. In recent years, advanced imaging and electrophysiological (AIE) methods have transformed our understanding of residual and covert conscious processing in patients who appear unresponsive at behavioral examination after severe brain injury. Multiple AIE techniques and paradigms have been developed to assess brain activity, and in some cases, detect preserved consciousness in patients that appear to be unresponsive. In this tutorial, an interdisciplinary group of experts will outline how state-of-the-art AIE techniques can be used to detect preserved brain activity at the individual subject level, in cases where the presence of sensory, motor and cognitive impairments might prevent the expression of consciousness. The tutorial is designed following a multimodal, integrated approach, where speakers will alternate and interact in presenting practical applications of all major techniques and paradigms available to assess brain activity in DoC, including electrophysiology (electroencephalography (EEG) alone and in combination with transcranial magnetic stimulation (TMS-EEG)), functional neuroimaging (functional magnetic resonance imaging (fMRI) and functional near-infrared spectroscopy (fNIRS)) and imaging of metabolic activity (positron emission tomography (PET)). The tutorial will be divided into three main parts. First, a theoretical session where we will provide a short introduction to DoC and the evolution of neuroimaging and electrophysiological assessment in DoC, followed by a presentation of each technique and the indices of consciousness that each method can provide. Second, an interactive hands-on session where real clinical cases, and the results obtained from a multimodal assessment (fMRI, fNIRS, PET, EEG, TMS-EEG), are critically discussed. Third, a demonstrative session where the participants will have the chance to be guided through a recorded TMS-EEG session, and through an overview of analysis techniques for TMS-EEG, EEG, and PET. The tutorial will be designed to be highly interactive, with Q&A times throughout the sessions. In session 1), the speakers will provide the attendees with hands-on guides and materials to run data processing and independently derive the indices of consciousness discussed during the tutorial. Active participation will be encouraged in sessions 2) and 3). Information on open-access resources to download freely accessible neurophysiological and neuroimaging datasets will also be provided. At the end of this tutorial, participants will have gained a general overview of how different paradigms can hierarchically detect various aspects of conscious processing and how AIE techniques can support diagnosis, prognosis, and guide rehabilitation. Attendees will also gain an understanding of the practical and technical implications of using AIE as well as the benefits and limitations of their application in DoC patients. We believe that the varying perspectives and insights from this tutorial will lead to increased knowledge and awareness of how advanced neuroimaging techniques can detect the neural correlates of consciousness in patients with DOC.

Because consciousness science suffers from language barriers with other fields, successfully applying for consciousness research funding can be a challenge. Communicating the excitement, rigor, and importance of consciousness science to the general public can prove harder still. These challenges in part stem from unfamiliarity and in part from general skepticism about consciousness science itself and its perceived overlap with fringe or pseudoscience, but this skepticism is significantly exacerbated – indeed, perhaps fueled – by difficulties in communicating the goals, approaches, and promise of consciousness research to scientists in related disciplines (perception, attention, neuroscience, medicine, AI, etc.) and to the educated general public due to specialized vocabularies. At worst, these communication barriers can result in a general perception that consciousness science is fundamentally questionable in its rigor; at best, they muddy outsiders’ understanding of our burgeoning and rigorous field, reduce probability that funding applications will be successful, and reduce the impact of scientific advances on basic science, medicine, ethics, and more

Through this tutorial, we hope to (1) help consciousness science researchers develop tools and language to effectively communicate about their research to people outside of their field, including funders; and (2) create connections and collaborations between participants as they brainstorm fundable consciousness-related projects and communication strategies together. Success in these goals is paramount to the field of consciousness science as it continues to proliferate and influence related fields such as perception, medicine, AI, and more. The time is ripe for a concerted and unified effort from the ASSC community to improve our collective communication for the benefit of our field, both with increased funding for empirical studies and accurate coverage of our exciting findings in public spheres.

The tutorial will include three core activities:

Tutorial presenters will share insights from their recent “Fund Consciousness Science” workshop (generously funded by the Templeton World Charity Foundation), which brought together early-career researchers in consciousness science and related disciplines for two days to discuss overlapping interests and expertise. The workshop also included senior researchers in consciousness science and nearby disciplines who had successfully competed for funding via mainstream mechanisms, as well as program officers from four United States based organizations to help guide and hone the messaging to be compelling and persuasive to a broader audience of scientists and grant reviewers from outside consciousness science. 

We will also reach out to funding agencies outside of the U.S. and add their input in order to make the tutorial useful for non-U.S. researchers.

We will also include presentations and insights on presenting consciousness science in written and oral form to non-specialist audiences, including public outreach. In light of the recent boom in coverage of consciousness research, science journalist Nora Bradford will also share tips for talking to journalists.

An interactive workshop component will group workshop attendees based on their answers to surveys distributed before the meeting. Small groups will discuss and brainstorm how they can hone their messaging to best convey their science to both funding agencies and the general public. 

In this tutorial, we will present the integrated information theory (IIT) of consciousness. The tutorial will be based on the recent paper, “Integrated Information Theory (IIT) 4.0: formulating the properties of phenomenal existence in physical terms” and its primary focus will be on the mathematical framework of the theory. There will be a brief introduction to the phenomenal axioms that form the basis of the theory, and their translation into physical postulates. For IIT, physical is meant operationally, as causes and effects that can revealed through manipulation and observation. IIT proposes an explanatory identify, that all aspects of subjective experience can be explained from the physical (cause-effect) properties of its substrate, with no additional ingredients or intrinsic properties of the units. Following the introduction, we will present IIT’s mathematical framework for quantifying cause-effect power, referring to the postulates throughout. The exposition will include simple “by hand” examples, as well as more detailed computational examples using the PyPhi software package. We will demonstrate how the mathematical framework can be used to explain why some substrates are conscious, while others are not (the quantity of experience), and why a substrate feels the way that it does (the quality of experience). Finally, we will conclude with a discussion of the explanations, predictions, and inferences that follow from the theory.

It is commonplace that philosophy and sciences collaborate with regard to consciousness, but the branch “philosophy of perception” has been developing in relatively isolated ways. Not only views such as naïve realism and adverbialism tend to be unintelligible for the scientists; philosophical discussions of sounds, smells, and individuating the senses tend to be neglected by the sciences. We hope to put up a tutorial that remedies the situation: in getting more scientists to be aware of such discussions in philosophy of perception, it will be helpful for interdisciplinary research about both perception and consciousness.

To this end, we will have four experts in the field of philosophy of perception, and they will sample some specific areas to introduce and lead discussions. The aim is not to be comprehensive; rather, we will make sure the selected topics of philosophy of perception can become intelligible to consciousness scientists so that meaningful intellectual interactions will become possible.

A. Spatial Perception and Multimodality

Tony Cheng

Are all sense modalities spatial? For those spatial ones, are they intrinsically spatial, i.e., at least some of their spatiality are not derived from other senses or faculties? We will begin by comparing sight and touch, by way of discussing Molyneux’s question, and extend our discussions to other senses, such as audition, olfaction, thermoception, etc. We will also cover multimodal perceptions such as synaesthesia and the ventriloquism effect.

B. Temporal Perception and the Self

Yuko Ishihara

It seems that all of our sense modalities are temporal, but their temporal profiles seem to differ significantly. There are many theoretical questions we can ask here: how do we perceive change? What can we understand about time itself from our perceptual experiences?  How can we understand the unity of consciousness over time? And how does the temporal existence of our consciousness relate to ourselves as persons or selves?

C. Perceiving Other Minds

John O’Dea

If we distinguish between features of the world that we perceive and features that we can only infer, it is natural to think that we infer the minds of others, rather than perceive them. But this has often been disputed in philosophy, especially in the past century, with increasing empirical support. On what grounds might one think that we can actually perceive the minds of others? In what sense? If minds can (in some sense) be perceived, how might this affect theories of consciousness?

D. Metaphysics of Perceptual Experiences

Takuya Niikawa

Many metaphysical theories on perceptual experiences, such as intentionalism and naive realism, have been proposed and developed among the community of philosophers. For scientists, however, it may seem unclear what they aim to explain, how they can be assessed, and how they are related to scientific understandings of perceptual consciousness. This part addresses those questions from the side of philosophers.

We will make sure to have enough breaks and discussion times in the session.

In this tutorial, we will introduce a range of different measures and computational models of metacognition, and provide a practical guide to fitting and comparing different models. The first part of the tutorial will cover the conceptual distinction between metacognitive sensitivity, efficiency and bias, and explore the pros and cons of different approaches to measuring these parameters. We will focus on the theory underpinning the meta-d’ model (a gold standard measure of metacognitive sensitivity), and its powerful and flexible hierarchical Bayesian extension (HMeta-d). 

In the second part of the tutorial we will explore next-generation computational models of confidence, and consider the relationship between simpler “measures” and more complex “models” of metacognition. We will introduce the models in an accessible way and discuss practical issues that arise when fitting computational models to behavioural data. We will step through code examples and build confidence amongst participants in being able to use these tools in their own projects.

The presenters are both active researchers in the field, leading research groups studying the computational and neural basis of metacognition. Of particular relevance to the current tutorial, Fleming is the lead developer of the HMeta-d toolbox, and Peters is developing new approaches to arbitrating between computational models of confidence. Versions of this tutorial has been delivered successfully at previous ASSC meetings (in 2017 in Beijing, in 2022 in Amsterdam and in 2023 in New York), at the Zurich Computational Psychiatry Course (in 2019 and 2021) and at the UCL-PSL Summer School on Consciousness and Metacognition (in 2021 and 2023). This year’s tutorial is comprehensively updated to encompass the fast-developing field of computational models of confidence. The accompanying code is thoroughly documented and freely available and will be a valuable resource for participants to apply meta-d’ analysis to their own data (https://github.com/metacoglab/HMeta-d/blob/master/CPC_metacog_tutorial/cpc_metacog_tutorial.m). To follow along with the code examples, participants will need a laptop with Matlab and JAGS installed.

By now, virtually all fields engaged in consciousness science have begun to address structural features of consciousness. Recognizing that even detailed verbal descriptions fall short of capturing the full depth and richness of conscious experience, structural approaches aspire to apply more holistic, precise and nuanced means of describing phenomenal character using mathematical structures or mathematical spaces. Structural ideas are expounded in great mathematical detail, are imputed to theories of consciousness, have become the central aim of many experiments on consciousness, are an increasing focus of philosophical theorising about consciousness, and are in the process of shaping what we pre-theoretically think about the problem of consciousness.

Yet, at the same time, there is no shared understanding of what the term ‘structure of consciousness’–or similar mathematical terminology–actually refers to. There are various ideas of what exactly terms like ‘structure’ or ‘space’ mean, over and beyond the question of which properties or modalities they target.  Much of contemporary research rests on intuitive understandings of these terms, or on reference to results implanted from psychophysics. The question of how one should understand or define the term ‘structure of consciousness’ is far from settled. 

In addition, even when a mathematical structure is rigorously defined, it remains non-trivial to further specify how exactly it translates to phenomenological terms or properties (the ‘Rosetta Stone Problem’ of David Chalmers).

The goal of this tutorial is to discuss and assess possible ways of understanding the terms ‘structure’ or ‘space’ when applied to conscious experience. The aim is to bring to the table a list of all major contemporary ways of defining structural terms, together with a clear emphasis of the respective merits, drawbacks, and connections. Fundamentally, the goal is to counteract the vagueness and ambiguity that veils the use of this terminology at present, so as to move towards a more informed, grounded and unified scientific programme.

Critically, the tutorial aims for an interactive format that includes participants in the exploration of these questions. 6 speakers will present their own views on the titular question in a short talk, leaving ample time for extended discussions of these proposals that make room for the participants’ expertise as well.

Ambiguities in understanding and conceptualising the core objects of structural research bring a risk of mutual misunderstanding, may misdirect effort, and in the worst case undermine a strong structural research programme. With this tutorial we hope to help resolve this fundamental issue, so as to support a rigorous science of consciousness.

Talks in this session:

1.) Joanna Szczotka – “What are Structural Properties of Consciousness? – Perspectives from IIT”

2.) Johannes Kleiner – “What is a Mathematical Structure of a Conscious Experience?”

3.) Chen Song – “Deconstruct and Reconstruct Consciousness: When the Part Conceals the Whole”

4.) Andrew Lee – “Structure vs Qualities”

5.) Lucia Melloni – “Phenomenal Structure and Phenomenal Spaces: What is That?”

6.) Sascha Benjamin Fink – “Limits and Prospects of Structural Approaches to Consciousness”

Since the launch of the search for the neural correlates of consciousness (NCCs), the field of consciousness science has made tremendous progress in identifying large-scale, network-level patterns of brain activity that are associated with consciousness and conscious perception. However, we lack even basic understanding of how such network-level patterns are supported by individual neurons and neural circuits. This gap is at least partially due to the fact that consciousness and conscious perception are difficult to study in experimental animals, where such detailed characterization of neural activity is possible. Such a gap in understanding between the macro and micro levels of consciousness-associated brain activity precludes findings from consciousness science from having maximal impact in clinical settings. Recently, both experimental and theoretical work have begun bridging this divide across spatial scales. In this tutorial, we bring together neuroscientists, philosophers, and engineers as well as theory, data, and neural modeling in order to provide an overview both of cellular-level theories of consciousness as well as a specific biophysical modeling tool – The Human Neocortical Neurosolver (HNN) – that permits direct testing of such theories by providing a multiscale cellular and circuit-level interpretation of human EEG/MEG. The tutorial will include four speakers: (1) Dr. Stephanie Jones (Brown University) will provide an introduction to biophysical modeling and HNN and an overview of how she and her colleagues have used HNN to examine conscious perception in the human somatosensory system using MEG. (2) Dr. Tomáš Marvan (Czech Academy of Sciences) will provide an overview of cellular-level theories of consciousness including both Apical Amplification Theory and Dendritic Integration Theory. (3) Dr. Andrew Dykstra (University of Miami) will present experimental and modeling results using HNN that demonstrate (i) the utility of biophysical modeling in the context of (auditory) conscious perception paradigms and (ii) the ability to test cellular-level theories of conscious perception using HNN. (4) Finally, Carolina Fernandez (University of Miami) and Stephanie Jones (Brown) will lead a practical, hands-on session that will demonstrate how to use HNN to study the neural circuit origin of source-localized event-related responses from either MEG or EEG recordings. Upon completion of the tutorial, attendees will have learned (i) the foundations of cellular-level theories of consciousness and conscious perception, (ii) the framework of biophysical modeling of human M/EEG data with HNN, and (iii) how to apply HNN towards their own data/questions concerning consciousness.

The experimental induction of altered states of consciousness (ASCs) constitutes a unique research opportunity to relate changes in phenomenological states to underlying neuronal mechanisms. A variety of pharmacological as well as non-pharmacological methods, such as breathing techniques and sensory deprivation, can induce ASCs in humans. Subjective reports suggest that ASCs, even when induced by different methods, share certain aspects of experiences. To clarify if shared subjective experiences also share neuronal mechanisms, an accurate psychometric assessment of subjects’ experiences is necessary. Multiple questionnaires have been developed based on qualitative reports and philosophical conceptualisations to quantify the phenomenology of ASCs. In this tutorial, we will present an overview on available psychometric tools, their theoretical background, and validation. We will discuss the questionnaires covering a broad range of different experiences in contrast to those that were designed to assess induction method specific effects, e.g., the effects typical to hallucinogens. Addressing a broad range of ASC experiences is required for the identification of common phenomenological structures of differently-induced ASCs. Based on their phenomenological scope and how much they have been used in previous studies, we will present recommendations for questionnaires to assess ASC phenomena in future neuroscientific experiments. Common standards for this rapidly extending body of research will foster comparability across different phenomenological states (‘phenomenological patterns’) and different studies. The comparison across studies represents an empirical framework to test how alterations in subjective experiences can be mapped onto brain functions and related to current theories on global brain function.

Whether one person’s experience of “red” is equivalent to another’s has long been considered unanswerable. A promising approach to address this fundamental question of consciousness is the intersubjective comparison of the similarity relations of sensory experiences, called “qualia structures”. Conventional methods for comparing similarity relations based on supervised alignment (e.g., Representational Similarity Analysis (RSA)) largely sidestep this issue, assuming that experiences elicited by the same stimuli are matched across individuals, and thus ruling out the possibility that my “red” could be your “blue”. Thus, supervised alignment methods are not suitable for assessing the equivalence or difference between the qualia structures of different individuals. Here, we propose instead to compare qualia structures based on unsupervised alignment, without assuming the correspondences between qualia across individuals, and taking into account all possibilities of correspondence.

In this tutorial, we present a practical and simple method of unsupervised alignment to compare qualia structures across individuals. This method is based on the Gromov-Wasserstein optimal transport, which finds the best mapping between qualia in terms of optimal transport based only on the internal relations of qualia structures. We have recently developed a user-friendly toolbox for the unsupervised alignment based on Gromov-Wasserstein optimal transport (https://github.com/oizumi-lab/GWTune). By using our toolbox, we aim to familiarize tutorial participants with unsupervised alignment and enable them to use the unsupervised alignment for applications of their use case.

In this tutorial, we step through the following unsupervised alignment procedures. First, psychological embeddings of qualia are estimated that best explain behavioral data such as similarity ratings or odd-one-out judgments. These psychological embeddings constitute qualia structures, where the distances between the embeddings are interpreted as the similarity between the corresponding qualia. Second, the estimated qualia structures are compared using the Gromov-Wasserstein optimal transport method. The optimization of the Gromov-Wasserstein distance can be efficiently performed by our toolbox. Third, based on the optimal transportation matrix, the psychological embeddings are aligned in a purely unsupervised manner. The aligned embeddings are visualized by principal component analysis. The quality of the unsupervised alignment is evaluated based on the matching rate when there is a known correspondence. 

After these procedures, participants will get hands-on experience with our toolbox. They will have the opportunity to analyze both open datasets and their own personal datasets. Finally, we will introduce other potential applications, such as the alignment of qualia structures with neural representation structures, and discuss future perspectives for consciousness research based on qualia structures and unsupervised alignment.

References

[1] Toolbox for Gromov-Wasserstein optimal transport: Application to unsupervised alignment in neuroscience

Masaru Sasaki, Ken Takeda, Kota Abe, Masafumi Oizumi

https://www.biorxiv.org/content/10.1101/2023.09.15.558038v1

[2] Is my “red” your “red”?: Unsupervised alignment of qualia structures via optimal transport

Genji Kawakita, Ariel Zeleznikow-Johnston, Ken Takeda, Naotsugu Tsuchiya, Masafumi Oizumi

https://osf.io/preprints/psyarxiv/h3pqm/

[3] Comparing Color Similarity Structures between Humans and LLMs via Unsupervised Alignment

Genji Kawakita, Ariel Zeleznikow-Johnston, Naotsugu Tsuchiya, Masafumi Oizumi

https://arxiv.org/abs/2308.04381