Project #1: Mobile Video Delivery System Design

Multimedia contents, especially videos, have seen their unprecedented growth of popularity in the last decades. The ultimate objective of various video delivery systems is to ensure that the end users can enjoy their best possible experiences. We believe we need to push for a significant departure from conventional quality-of-service paradigm and establish a new paradigm in terms of end-user-oriented system design that moves beyond core delivery system itself to encompass new issues of context and environments in mobile video to achieve best quality-of-experience.

The influence factors (IF’s) for video delivery include both systems IF’s as well as context IF’s and user IF’s as illustrated below. From system design perspectives, the end-to-end ecological chain includes host, channels, and clients.

Chen’s research team is working various topics related mobile video delivery, ranging from video quality assessment of user generated videos when the original videos are unavailable as reference, to mobile video viewing when the ambient environments changes dynamically and requires adaptation, to IoVT applications when the needs for designing new transmission strategy for 5G and future generation mobile communication architecture, and to designing power saving display of video for battery-operated mobile devices such as smart phone. Such a new design paradigm encompasses issues related to either the host-channels-client partition or the system-context-user partition.

Illustration of a novel pseudo-reference image assessment principle for the contemporary user-generated multimedia content delivery when the original reference images are unavailable. This is paradigm-shifting idea in that, instead of comapring the image to perfect quality, the pseudo reference is estimated from possible worst distoritons. This idea can be relatively easily implemented as the following operational pipelines:

Conventional video perception focuses on the outcomes from controlled laboratory viewing environment for living room settings. As the smart phones have now become the primary platform for viewing videos, the environments in which the consumers viewing the video signal have changed completely. It has been recognized that three new primary viewing contextual environments have been identified: viewing distance, ambient environment, and viewer motion patterns. It is necessary to design new adaptation schemes based on viewing context environments.

New generation of smart phones have been equipped with various sensors whose data can be utilized to estimate the viewing contextual environments. We have designed an novel adaptation scheme to maximize smart phone users’ mobile video perception experiences.

Battery power issues are extremely important for mobile devices especially when the mobile devices such as smart phones are now frequently used to view video contents on the go. It is well-known that the brightness level of the mobile display determines how quickly the battery would be drain out. Coupled with the wireless transmission of the video content, the video delivery to mobile devices service consumes 80% or more power when such service is in session. There are various on-board strategies to save the power consumption. Many of these schemes either consume additional power in terms of video analytics or cause undesired degradation of the video quality. We have developed a holistic approach to attack such a systematic problem by shifting the computation burden from video analytics to cloud center and piggy-pack required operational parameters for display adaptation via negligible data augmentation. This is accomlished by an innovative solution to obtain an R-D-DE profile for a given video content.

The above illustration shows the paradigm shifting idea by augmenting the traditional rate-distortion (R-D) principle to the new rate-distortion-display energy (R-D-DE) principle to incorporate the display energy reduction (DER) into the overall system design. This augmentation enables the shifting of per-device computation on each smart phone to cloud-based computation to save the smart phone power at a massive scale. Every smart phone that access the video content processed by R-D-DE mechanism will save the operation of video content analysis for DER operation. We have also designed several energy saving schemes for video viewing services under various viewing conditions.