AMCA 2022 Summer School Report

By Federica D’Antoni, Sofian A. Bouaouina, Laurent Camus, Thomas Debois, Guillaume Gauthier, Philipp Hänggi, Mizuki Koda, Yeji Lee, Lorenza Mondada, Julia Schneerson, Hanna Svensson 

Hosted by the University of Basel and the Hermann Paul School of Linguistics, the Advancing Multimodal Conversation Analysis (AMCA) Summer School took place from the 21^st to the 24^th of June, 2022, in Basel, Switzerland. Targeting early career researchers with an advanced understanding of Multimodal Conversation Analysis (CA), the summer school provided a platform for participants to discuss a myriad of issues and challenges concerning Multimodal CA with leading researchers in the field. 

A Multimodal Turn in CA 

With the purpose of advancing Multimodal CA, the summer school tackled both conceptual matters as well as methodological issues characterizing current CA reflections on language and the body in social interaction and the bearing they have on the observability of specific social phenomena. John Heritage (UCLA) and Anita Pomerantz (University at Albany) reiterated the quintessential roles of action ascription and sense-making work intrinsic to any interactional conduct. Heritage argued that people make use of a diversity of resources when they engage in action ascription, e.g., sequential positioning within sequences and activities, the social-institutional context, personal statuses that people attribute to each other, and the rights and obligations that come with them. Pomerantz proposed a reflection on how shared assumptions about and within an interaction are a demonstrable, intersubjective achievement. By analyzing assessments doing self-praising and disagreeing with prior assessments, and instances where questions about prior experiences emerge, she showed how people use strategies to display their expectancies regarding what is relevant to be done next. In this way, they both discussed the role of shared assumptions and contextual circumstances in action formation and in the organization of recognizable social actions, showing how social phenomena can be approached by going beyond a cognitivist framework. An alternative framework is a multimodal approach to social actions, or Multimodal CA, where the accountability of action is enhanced by embodied details, eventually expanding the list of ‘resources’ that compose social interaction. Aug Nishizaka (Chiba University) gave a candidate example of such expansion of multimodal resources, as he addressed the situatedness of perception in interaction. Nishizaka discussed how doing “seeing” and “inspecting” objects in particular ways is treated as constitutive of particular actions (as opposed to an exclusively physiological process of collecting visual information), and how ways of looking make relevant particular features of objects and actions. 

Building Collections of Multimodal Phenomena 

A substantial part of the summer school was dedicated to challenges and issues that researchers encounter when building collections of phenomena within Multimodal CA. Jeffrey Robinson (Portland State University) offered detailed insight on this methodological step, proposing some guidelines for the analyst to keep in mind when collecting instances of social actions, or nuances of them. Robinson notably addressed the problem of granularity and the possibilities and limitations that we might encounter when pursuing a more detailed focus on social phenomena. 

We also deepened our understanding of Multimodal CA and practiced its use in the workshops. Burak S. Tekin (Ankara Yıldırım Beyazıt University) convened the first workshop where he presented his work on accounting for the body in Kinect video-game play activity. His lecture was followed by group work dedicated to creating candidate collections. Tekin concluded by discussing the challenges of building collections, particularly highlighting the multitude of ways in which phenomena can be formulated and its consequences on how collections are built. The second workshop was organized by Florence Oloff (IDS Mannheim) who gave a presentation on smartphone-based showing sequences, emphasizing how multimodal phenomena (especially assessments) are embedded in the temporality and sequentiality of the interaction. The lecture was followed by group work where participants worked together to build candidate collections of smartphone-based showing sequences. Oloff underscored the importance of considering the multiple multimodal features of an action in their complexity (i.e., the local ecology, the materiality of smartphones) as a reference point for building collections on multimodal phenomena.

Addressing Complex Settings and Activities 

Another big part of the summer school was spent on examining data from complex settings and activities and on how embodied actions are configured in those interactions. One perspicuous setting to appreciate the complexity of social actions proved to be technology-based/assisted activities. Through Oloff’s workshop on smartphone-based showing sequences, the materiality of smartphones and the temporality in and of their use were highlighted. On the one hand, the material features of a smartphone, such as its size and screen orientation, became resources for organizing the interactional space among participants. On the other hand, participants in the data were seen to orient to the temporality of using a smartphone such as pausing or resuming what is played on the screen to coordinate the progressivity of their ongoing talk with it. 

Likewise, Tekin’s workshop was based on the use of Kinect technology in video-game play data, and the main task was to identify and formulate actions that occupy these activities. In this setting, players’ actual bodies are represented as avatars’ bodies on the screen. Playing the game appropriately, therefore, entails that players make their bodies recognizable to the sensor on the screen. The complexity of this setting where both players and non-players (or spectators) engage in making sense of the technological apparatus prompted discussions on the organization of participation frameworks. One of the key ways by which participation was organized was in how participants in the interaction accounted for the players’ bodies in reference to the bodies on the screen. 

Another complex setting that proved fruitful for the investigation of participation frameworks was shop encounters at the market, as introduced by the research team in Basel (University of Basel) who organized the third workshop. A practical problem for sellers and customers at shops is to manage multiple participation frameworks when there are several customers or to-be customers that need to be recognized and monitored by a smaller number of sellers. We analyzed the micro-adjustments of embodiment of the parties, and the way they mutually oriented to each other while orderly dealing with several courses of actions at the same time. This workshop not only allowed us to discuss the multimodal organization of multiple participation frameworks but initiated a reflection on the importance of multimodal transcription for discovering and representing the complex intertwinement of talk and body. 

A Hybrid, Inclusive, and Hands-On Model of Training 

The summer school was original in its hybrid, inclusive, as well as hands-on model of training. The hybrid format has been on the rise since the advent of the pandemic, and we also adopted it so that invited speakers from the US and Japan could give lectures while also making those lectures accessible to the larger international EMCA community who could not attend in person. We opted for a format of hybridization that consisted in making the lectures available to the public but leaving the discussions afterward exclusive to on-site participants and speakers. In so doing, we could provide a safer and more intimate environment for early career researchers to engage in extensive discussions with more advanced scholars, something that could be more challenging in a larger participation framework on Zoom. All in all, this ‘glocal’ hybridized format worked very well, and we would encourage other organizing committees to introduce and experiment with such variations. 

One of the most important factors we considered when designing the summer school was to make it maximally inclusive. First and foremost, we made sure high registration costs did not stop researchers in their tracks. Therefore, we kept the costs for the 4-day summer school as low as CHF/EUR 60.00 while providing most meals—many were saying that there was just so much good food! Another critical aspect of inclusiveness was to provide the larger EMCA community with free access to remote lectures. At the discretion of the speakers, we set up a Zoom Webinar link so that anyone interested, including those who were not registered at the school, could join and listen in.

Another organizational pillar that was dear to us was placing special emphasis on group work. Throughout several hands-on workshops, the registered participants could intensively engage with data provided by workshops’ leaders. Small group work in the form of two or three participants per group enabled participants to collectively discuss several analytical tasks—ranging from initial ‘unmotivated looking’ at data to identifying, systematically transcribing, and ultimately presenting candidate multimodal phenomena—head-on, encouraging them to develop their analytical skills and get more familiar with the ins and outs of multimodality. All things considered, this intense hands-on confrontation with Multimodal CA turned out to be a fertile ground for compelling debates as well as discussions, be it within the groups or in the ensuing plenum discussion(s). 

Moving Forward 

From the rich and varied experiences of this summer school, we conclude that there is still much to explore on the topic of Multimodal CA. The field is burgeoning with new types of data, recording techniques, social situations and activities, relying on the body, language, technologies, and materialities. The field, however, also needs to consolidate fundamental assumptions: based on feedback from participants, one complementary direction for an upcoming school includes more focus on multimodal transcription in terms of how consequential its use is for certain types of analyses where the notion of ‘relevance’ remains key. Training the detailed practice of multimodal transcription is what makes it possible for analysts to look at more difficult and extreme boundary cases which are challenging to transcribe, and to reflect on what it means to have neither formalistic/physiological nor intentionalistic approaches to embodiment. Multimodal CA is undoubtedly a promising field with many potential directions for future research. A lot remains to be explored – we, therefore, see great value and relevance for Multimodal CA as a topic and resource that seriously considers embodiment in the organization of sociality and intersubjectivity in everyday life.