This book integrates a wide range of research topics related to and necessary for the development of proactive, smart, computers in the human interaction loop, including the development of audio-visual perceptual components for such environments; the design, implementation and analysis of novel proactive perceptive services supporting humans; the development of software architectures, ontologies and tools necessary for building such environments and services, as well as approaches for the evaluation of such technologies and services.

The book is based on a major European Integrated Project, CHLI (Computers in the Human Interaction Loop), and throws light on the paradigm shift in the area of HCI that rather than humans interactive directly with machines, computers should observe and understand human interaction, and support humans during their work and interaction in an implicit and proactive manner.

This book integrates a wide range of research topics related to and necessary for the development of proactive, smart, computers in the human interaction loop, including the development of audio-visual perceptual components for such environments; the design, implementation and analysis of novel proactive perceptive services supporting humans; the development of software architectures, ontologies and tools necessary for building such environments and services, as well as approaches for the evaluation of such technologies and services.

The book is based on a major European Integrated Project, CHLI (Computers in the Human Interaction Loop), and throws light on the paradigm shift in the area of HCI that rather than humans interactive directly with machines, computers should observe and understand human interaction, and support humans during their work and interaction in an implicit and proactive manner.

This book constitutes the thoroughly refereed joint post-workshop proceedings of two co-located events: the Second International Workshop on Classification of Events, Activities and Relationships, CLEAR 2007, and the 5th Rich Transcription 2007 Meeting Recognition evaluation, RT 2007, held in succession in Baltimore, MD, USA, in May 2007.

The workshops had complementary evaluation efforts; CLEAR for the evaluation of human activities, events, and relationships in multiple multimodal data domains; and RT for the evaluation of speech transcription-related technologies from meeting room audio collections. The 35 revised full papers presented from CLEAR 2007 cover 3D person tracking, 2D face detection and tracking, person and vehicle tracking on surveillance data, vehicle and person tracking aerial videos, person identification, head pose estimation, and acoustic event detection. The 15 revised full papers presented from RT 2007 are organized in topical sections on speech-to-text, and speaker diarization.

TheseriesofworkshopsonMachineLearningforMultimodalInteraction(MLMI) celebratesthisyearits?fthanniversary.Onthisoccasion, anumberofinnovations havebeenintroducedin the reviewingandpublicationprocedures, while keeping the focus onthe samescienti?c topics. For the ?rst time, the reviewing process has been adapted in order to p- parethe proceedings in time for the workshop, held on September 8-10,2008, in Utrecht, The Netherlands. The 47 submissions received by the Program C- mittee were ?rst reviewed by three PC members each, and then advocated by an Area Chair. Overall, 12 oral presentations (ca. 25% of all submissions) and 15 poster presentations were selected. Authors were given one month to revise their papers according to the reviews, and the ?nal versions were brie?y checked by the two Program Co-chairs. Both types of presentation have been give equal space in the present proceedings. The 32 papers gathered in this volume cover a wide range of topics - lated to human-human communication modeling and processing, as well as to human-computer interaction, using several communication modalities. A sign- icant number of papers focus on the analysis of non-verbal communication cues, such as the expression of emotions, laughter, face turning, or gestures, which demonstrates a growing interest for social signal processing. Yet, another large set of papers targets the analysis of communicative content, with a focus on the abstractionofinformationfrommeetingsintheformofsummaries, actionitems, ordialogueacts.OthertopicspresentedatMLMI2008includeaudio-visualscene analysis, speech processing, interactive systems and applica