Activity Recognition and Psychophysics: Towards Real-Time Processing of 3D Tele-immersive Video
Klara Nahrstedt, 09:15-10:30, Session Chair: Laszlo Böszörmenyi
With the decreasing cost and ubiquity of 3D cameras, real-time 3D tele-immersive video is becoming a real possibility in the Next Generation of stored and interactive teleimmersive systems. However, real-time processing of 3D tele-immersive videos is still very challenging when executing queries, transmission and/or user interactions with 3D tele-immersive video. In this talk, we will make the case that in order to achieve real-time processing of 3D tele-immersive video, we need to consider two important issues, the understanding of the user perception, and detection of semantics. Based on different case studies, we discuss two psychophysical activity-driven approaches for real-time 3D teleimmersive video. Both approaches achieve real-time processing due to adaptive compression of 3D teleimmersive videos based on understanding of user perception and detecting activities semantics in the content. Both approaches reduce underlying resource usage, while preserving the overall perceived visual Quality of Experience.
In the first case study, we show the importance of the semantic factor “CZLoD: Color-plus-Depth Level-of-Details”. Through psychophysical study we show the existence of two important thresholds, the Just Noticeable Degradation and Just Unacceptable Degradation thresholds on the CZLoD factor which are activity-dependent. This approach then utilizes the activity-dependent CZLoD thresholds in the real-time perception-based quality adaptor, while reducing underlying resource usage and enhancing perceived visual quality. In the second study, we combine activity recognition and real-time morphing-based compression of 3D tele-immersive video, where the morphing rate is controlled by user experience and activities within a resource adaptor, reducing bandwidth and preserving perceived visual quality. In both studies, the results are very encouraging, promising fast retrieval and transmission times, and high quality user interactions with 3D teleimmersive video.
Klara Nahrstedt is the Ralph and Catherine Fisher Professor in the Computer Science Department, and Acting Director of Coordinated Science Laboratory in the College of Engineering at the University of Illinois at Urbana-Champaign. Her research interests are directed toward 3D teleimmersive systems, mobile systems, Quality of Service (QoS) and resource management, Quality of Experience in multimedia systems, and real-time security in mission-critical systems. She is the co-author of widely used multimedia books `Multimedia: Computing, Communications and Applications’ published by Prentice Hall, and ‘Multimedia Systems’ published by Springer Verlag. She is the recipient of the IEEE Communication Society Leonard Abraham Award for Research Achievements, University Scholar, Humboldt Award, IEEE Computer Society Technical Achievement Award, and the former chair of the ACM Special Interest Group in Multimedia. She was the general chair of ACM Multimedia 2006, general chair of ACM NOSSDAV 2007 and the general chair of IEEE Percom 2009.
Klara Nahrstedt received her Diploma in Mathematics from Humboldt University, Berlin, Germany in numerical analysis in 1985. In 1995 she received her PhD from the University of Pennsylvania in the Department of Computer and Information Science. She is ACM Fellow, IEEE Fellow, and Member of the Leopoldina German National Academy of Sciences.