This volume advances the data-based study of multimodal artefacts and performances by showcasing methods and results from the latest endeavors in empirical multimodal research, representing a vibrant international and interdisciplinary research community. The collated chapters identify and seek to inspire novel, mixed-method approaches to investigate meaning-making mechanisms in current communicative artifacts, designs, and contexts; while attending to their immersive, aesthetic, and ideological dimensions. Each contribution details innovative aspects of empirical multimodality research, offering insights into challenges evolving from quantitative approaches, particular corpus work, results from eye-tracking and psychological experiments, and analyses of dynamic interactive experiences. The approaches and results presented foreground the inherent multidisciplinary nature and implications of multimodality, renegotiating concepts across linguistics, media studies, (social) semiotics, game studies, and design. With this, the volume will inform both current and future developments in theory, methods, and transdisciplinary contexts and become a landmark reference for anyone interested in the empirical study of multimodality.

Multimodality's popularity as a semiotic approach has not resulted in a common voice yet. Its conceptual anchoring as well as its empirical applications often remain localized and disparate, and ideas of a theory of multimodality are heterogeneous and uncoordinated. For the field to move ahead, it must achieve a more mature status of reflection, mutual support, and interaction with regard to both past and future directions. The red thread across the disciplines reflected in this book is a common goal of capturing the mechanisms of synergetic knowledge construction and transmission using diverse forms of expressions, i.e., multimodality. The collection of chapters brought together in the book reflects both a diversity of disciplines and common interests and challenges, thereby establishing an excellent roadmap for the future. The contributions revisit and redefine theoretical concepts or empirical analyses, which are crucial to the study of multimodality from various perspectives, with a view towards evolving issues of multimodal analysis. With this, the book aims at repositioning the field as a well-grounded scientific discipline with significant implications for future communication research in many fields of study.

Multimodality is one of the most popular and influential semiotic theories for analysing media. However, the application and conceptual anchoring of multimodality often remains geographically and disciplinarily grounded within local systems of thought. New Studies in Multimodality combines the expertise of multimodalists from around the globe, offering novel readings and applications of central concepts in multimodality and inviting innovative synergies between previously disparate schools. Combining perspectives from the most actively developing traditions of theory and research, this book progresses from classic concepts to more empirically and practice-motivated contributions. Contributors engage in mutual dialogue to present new theoretical perspectives and compelling applications to a variety of old and new media. Expanding the basis and scope of multimodality, this volume shows awareness and experience of this field in many disciplines and illustrates how versatile, pervasive and relevant it is for studying today's communication phenomena.

This edited volume brings together work in the field of empirical comics research. Drawing on computer and cognitive science, psychology and art history, linguistics and literary studies, each chapter presents innovative methods and establishes the practical and theoretical motivations for the quantitative study of comics, manga, and graphic novels. Individual chapters focus on corpus studies, the potential of crowdsourcing for comics research, annotation and narrative analysis, cognitive processing and reception studies. This volume opens up new perspectives for the study of visual narrative, making it a key reference for anyone interested in the scientific study of art and literature as well as the digital humanities.

This book examines film as a multimodal text and an audiovisual synthesis, bringing together current work within the fields of narratology, philosophy, multimodal analysis, sound as well as cultural studies in order to cover a wide range of international academic interest. The book provides new insights into current work and turns the discussion towards recent research questions and analyses, representing and constituting in each contribution new work in the discipline of film text analysis. With the help of various example analyses, all showing the methodological applicability of the discussed issues, the collection provides novel ways of considering film as one of the most complex and at the same time broadly comprehensible texts.

This book is a first attempt to map the broad context of performance studies from a multimodal perspective. It collects original research on traditional performing arts (theatre, dance, opera), live (durational performance) and mediated/recorded performances (films, television shows), as well as performative discursive practices on social media by adopting several theories and methodologies all dealing with the notion of multimodality. As a mostly dynamic and also interactive environment for various text types and genres, the context of performance studies provides many opportunities to produce meaning verbally and non-verbally. All chapters in this book develop frameworks for the analysis of performance-related events and activities and explore empirical case studies in a range of different ages and cultures. A further focus lies on the communicative strategies deployed by different communities of practice, taking into account processes of production, distribution, and consumption of such texts in diverse spatial and temporal contexts.

This book contributes to the analysis of film from a multimodal and textual perspective by extending formal semantics into the realm of multimodal discourse analysis. It accounts for both the inferential as well as intersemiotic meaning making processes in filmic discourse and therefore addresses one of the main questions that have been asked within film theory and multimodal analysis: How do we understand film and multimodal texts? The book offers an analytical answer to this question by providing a systematic tool for the description of this comprehension process. It aims to advance knowledge of the various resources in filmic texts, the ways the resources work together in constructing meaning and the ways people understand this meaning construction. This new approach to film interpretation is thus able to remodel and improve the classical paradigm of film text analysis.

This book contributes to the analysis of film from a multimodal and textual perspective by extending formal semantics into the realm of multimodal discourse analysis. It accounts for both the inferential as well as intersemiotic meaning making processes in filmic discourse and therefore addresses one of the main questions that have been asked within film theory and multimodal analysis: How do we understand film and multimodal texts? The book offers an analytical answer to this question by providing a systematic tool for the description of this comprehension process. It aims to advance knowledge of the various resources in filmic texts, the ways the resources work together in constructing meaning and the ways people understand this meaning construction. This new approach to film interpretation is thus able to remodel and improve the classical paradigm of film text analysis.

This edited volume brings together work in the field of empirical comics research. Drawing on computer and cognitive science, psychology and art history, linguistics and literary studies, each chapter presents innovative methods and establishes the practical and theoretical motivations for the quantitative study of comics, manga, and graphic novels. Individual chapters focus on corpus studies, the potential of crowdsourcing for comics research, annotation and narrative analysis, cognitive processing and reception studies. This volume opens up new perspectives for the study of visual narrative, making it a key reference for anyone interested in the scientific study of art and literature as well as the digital humanities.

This book examines film as a multimodal text and an audiovisual synthesis, bringing together current work within the fields of narratology, philosophy, multimodal analysis, sound as well as cultural studies in order to cover a wide range of international academic interest. The book provides new insights into current work and turns the discussion towards recent research questions and analyses, representing and constituting in each contribution new work in the discipline of film text analysis. With the help of various example analyses, all showing the methodological applicability of the discussed issues, the collection provides novel ways of considering film as one of the most complex and at the same time broadly comprehensible texts.

This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today's increasingly complex combinations of multimodal artefacts and performances of all kinds.

While multimodality is one of the most influential semiotic theories for analysing media artefacts, the concepts of this theory are heterogeneous and widespread. The book takes the differences between approaches in Germany and those in international contexts as a starting point, offering new insights into the analysis of multimodal documents. It features contributions by researchers from more than 15 nations and various disciplines, including theoretical reflections on multimodality, thoughts about methodological, empirical, and experimental approaches as well as analyses of various multimodal artefacts.

Multimodality is one of the most popular and influential semiotic theories for analysing media. However, the application and conceptual anchoring of multimodality often remains geographically and disciplinarily grounded within local systems of thought. New Studies in Multimodality combines the expertise of multimodalists from around the globe, offering novel readings and applications of central concepts in multimodality and inviting innovative synergies between previously disparate schools. Combining perspectives from the most actively developing traditions of theory and research, this book progresses from classic concepts to more empirically and practice-motivated contributions. Contributors engage in mutual dialogue to present new theoretical perspectives and compelling applications to a variety of old and new media. Expanding the basis and scope of multimodality, this volume shows awareness and experience of this field in many disciplines and illustrates how versatile, pervasive and relevant it is for studying today's communication phenomena.