Keynote Speakers
Elisabeth Andre
Universität Augsburg | Webpage
Title:
Embodied Multimedia: From Multimedia Content to Situated Human-AI InteractionAbstract:
Multimedia research, which has traditionally been concerned with the analysis and generation of multimedia content, has increasingly addressed multimodal human behavior. This development is exemplified by the MultiMediate Grand Challenge Series, which promotes multimodal behavior understanding across diverse social and cultural settings. The next frontier is to develop embodied AI systems that understand, support, and seamlessly participate in situated human interaction. We pursue this vision through virtual agents and social robots that facilitate communication between people and AI and increasingly between people themselves. To this end, we demonstrate how pretrained multimodal foundation models can be leveraged to create context-aware embodied AI. Drawing on applications in assistive communication, AI-mediated group discussions, and child therapy, the talk will illustrate how embodied agents enabled by these models can help overcome communication barriers, facilitate social interaction, and foster authentic self-expression, while highlighting the scientific challenges of grounding AI in the physical and social world.Bio:
Elisabeth André (Senior Member, IEEE) is a full professor of computer science and founding chair of human-centered artificial intelligence with Augsburg University, Germany. She has a long track record in multimodal human-machine interaction, embodied conversational agents, social robotics, affective computing and social signal processing. Her work has won many awards including the Gottfried Wilhelm Leibniz Prize, the most important research funding award in Germany. She is an elected member of the Academy of Europe, the German Academy of Sciences Leopoldina, the National Academy of Science and Engineering (acatech), the Bavarian Academy of Sciences and Humanities (BadW), and the CHI Academy. She is also a fellow of EurAI (European Association for Artificial Intelligence), AAAC (Association for the Advancement of Affective Computing), and ELLIS (European Laboratory for Learning and Intelligent Systems). In addition, she is a Golden Core Awardee of the IEEE Computer Society. In 2019, she was named one of the 10 most influential figures in the history of AI in Germany by National Society for Informatics (GI). From 2019 to 2022, she was serving as the editor-in-chief of IEEE Transactions on Affective Computing.
Haizhou Li
The Chinese University of Hong Kong | Webpage
Title:
Listening the Chinese WayAbstract:
Humans have a remarkable ability to pay their auditory attention only to a sound source of interest, that we call selective auditory attention, in a multi-talker environment or a Cocktail Party. As discovered in neuroscience and psychoacoustics, the auditory attention is achieved by a modulation of top-down and bottom-up attention. However, signal processing approach to target speaker extraction in a multi-talker environment remains a challenge. Human listening is more of an art than a science. The Chinese character for the word ‘listening’ carries the ancient wisdom about important aspects of what people believe it means to really ‘listen’ to someone, i.e. hearing, seeing, and understanding. In this talk, we study the deep learning solutions to target speaker extraction that enable selective auditory attention. We review the findings from human audio-visual speech perception to motivate the design of speech perception algorithms. We will also discuss the computational auditory models, technical challenges and the recent advances in the field.
Bio:
Dr. Haizhou Li is a Presidential Chair Professor and Associate Dean (Research) at the School of Data Science, CUHKSZ, and is also with NUS. He earned his B.Sc., M.Sc., and Ph.D. from South China University of Technology. A veteran in speech and language technology since 1988, Dr. Li has held global academic posts and significant research/industry leadership roles, including at Apple-ISS Research Centre, Lernout & Hauspie, and the Institute for Infocomm Research (I2R), Singapore. He co-founded the Baidu-I2R Research Centre. His technical contributions include award-winning products like Apple’s Chinese Dictation Kits and major deployments like TELEFIQS voice service and Baidu Music Search. His research interests are ASR (Automatic Speech Recognition), NLP (Natural Language Processing), and IR (Information Retrieval). He has held numerous prestigious service positions, including Editor-in-Chief of IEEE/ACM TASLP (Transactions on Audio, Speech and Language Processing) and President of ISCA (International Speech Communication Association) and APSIPA (Asia Pacific Signal and Information Processing Association). He has chaired major conferences like ACL (Association for Computational Linguistics) 2012, INTERSPEECH 2014, and ICASSP (International Conference on Acoustics, Speech, and Signal Processing) 2022. Dr. Li is an IEEE Fellow and an ISCA Fellow, and a recipient of Singapore’s President’s Technology Award (2013).
Clarisse Sieckenius de Souza
Pontifícia Universidade Católica do Rio de Janeiro | Webpage
Title:
Who Speaks for the Source? Semiotic Engineering, Delegated Reading, and the Multimedia Summarization ProblemAbstract:
Every system interface is a message: designers speak to users through what they build, telling them — implicitly or explicitly — who they think the user is and how the system can be used to meet their needs. Semiotic engineering calls this metacommunication, and the system that carries it the designers' deputy. Because it positions designers and developers as first-class citizens in an ongoing process of computer-mediated social communication, semiotic engineering has, since its inception, been natively engaged in ethical reflection about technology — not as an add-on, but as a consequence of taking seriously who speaks to whom, and on whose behalf, through what we build. This is a direct theoretical line: from its origins in human-computer interaction, through its later extension into human-centered computing, semiotic engineering's commitment to people on both sides of the screen leaves it well prepared to engage the ethics of AI and science — a debate it arrives at not as a newcomer, but as a natural continuation of its founding concerns. AI summarization quietly breaks the metacommunicative bargain that gives a deputy its accountability. When a researcher delegates the reading of source material to an AI system, the output has every surface feature of a message from a responsible sender — fluent, confident, addressed to a reader — but there is no principal behind it: no accountable agent has actually read the source in the sense that licenses testimony, and no "thing" or "person" can be held responsible for what it omits. I call this the delegated reading problem. Our Protocols for Epistemically Engaged Literacy in AI (PEEL) follows semiotic engineering's vocational road in one particular direction — scientific research, arguably among the most consequential arenas in which AI, left unharnessed, can do humanity good or harm. PEEL tackles summarization, a small but telling piece of that iceberg, giving researchers tools to audit what an AI did to a text rather than trust it outright. I close by turning the problem over to this community: what does an accountable deputy look like when the source is video, audio, or a multimodal stream?
Bio:
Clarisse Sieckenius de Souza is an Emeritus Professor at PUC-Rio, where she was a Full Professor in the Department of Informatics for over 30 years, specializing in Artificial Intelligence (AI) and Human-Computer Interaction (HCI). She holds Bachelor, Master, and Ph.D. degrees in Letters and Applied Linguistics/Computational Linguistics from PUC-Rio. She completed postdoctoral research at Stanford University and the University of Maryland, along with numerous visiting roles at institutions like IBM Research and Philips Research Labs. Her main contribution to HCI is the development of Computational Semiotics, specifically the concept of Semiotic Engineering proposed in 1993. In 1996, she founded SERG (Semiotic Engineering Research Group) at PUC-Rio, internationally recognized for developing the first semiotic theory of HCI. She is the author or co-author of four key books on Semiotic Engineering. Dr. de Souza is a reference in her field, receiving the ACM SIGCHI CHI Academy Award (2013), the IFIP HCI Pioneer Award (2014), and the SBC (Brazilian Computer Society) Scientific Merit Award (2016). She was named a "Notable Woman in Computing" in 2014. She is recognized as one of the founders of the HCI field in Brazil, receiving the SBC Career Achievement Award in 2017. As an Emeritus Professor, her current research focuses on ethics and algorithmic mediation and the use of digital technology in academic knowledge production.
Pamela Wisniewski
International Computer Science Institute (ICSI), UC Berkeley | Webpage
Title:
Safety by Design: Building a Safer Multimedia World for Young PeopleAbstract:
Drawing on more than a decade of research on adolescent online safety, this talk highlights how the multimedia community can move beyond risk mitigation toward safety by design. Adolescents’ online lives unfold largely through social media and other rich multimodal environments, where text, images, video, audio, behavioral signals, and social relationships combine to create both meaningful opportunities and serious risks. Yet, many online safety technologies reduce this complexity to a narrow classification problem: detect harmful content, flag a risky interaction, or restrict access altogether. Such approaches can miss the relational and developmental context in which harm and benefits occur and may unintentionally undermine young people’s privacy, autonomy, and willingness to seek help. Cybergrooming and online sexual risk on social media platforms will serve as a motivating case, illustrating why risk cannot be inferred from a single message or modality, but must be understood within the context of evolving conversations, including both visual and linguistic cues, power dynamics, and the perspectives of young people themselves. This talk will challenge the multimedia community to consider not only what we are technically capable of building, but also what we should build, for whom, and with what consequences. I argue for sociotechnical systems that build resilience, preserve meaningful social connection, and engage young people—especially those most vulnerable to online harm—as design partners. I conclude by considering how to build responsible multimodal AI systems that are context-aware, privacy-preserving, trauma-informed, and equipped with safeguards that protect, rather than constrain, the young people whose lives they will ultimately shape.