360° Video Streaming to Head-Mounted Displays

Watching a live broadcast of an event, such as a football match or a music concert, with traditional planar televisions or monitors is a passive experience. Over the past years, Virtual Reality (VR) products, such as Head-Mounted Displays (HMDs), become widely available. Many companies in the computer industry release their HMDs, such as Oculus Rift DK2, HTC Vive, and Samsung Gear VR. These products offer viewers wider Field-of-Views (FoVs) and provide more immersive experience than traditional ones. Besides, lots of 360° cameras are also introduced. For instance, Ricoh Theta S, Luna 360 VR, and Samsung Gear 360. With the growing popularity of consumer VR products, viewers are able to watch 360° videos. On top of that, major multimedia streaming service providers, such as Facebook and YouTube all support 360° video streaming for VR content.

Nowadays, majority of global Internet traffic is due to video data (Global Internet Phenomena). Streaming 360° videos further increases the Internet traffic amount, and becomes a hot research topic. Leveraging commodity HMDs for 360° video streaming is, however, very challenging for two reasons. First, 360° videos, illustrated in Fig. 1(a), contain much more information than conventional videos, and thus are much larger in resolutions and file sizes. Fig. 1(b) reveals that, with HMDs, each viewer only gets to see a small part of the whole video. Therefore, sending the whole 360° video in full resolution may lead to waste of resources, such as network bandwidth, processing power, and storage space. Another way to stream 360° videos to HMDs is to only stream the current FoV of the viewers. We emphasize current, because the FoV changes as the viewer’s head and eyes move, which leads to the following main challenge: which FoV should we transfer to meet the viewer’s needs in the next moment? This challenge makes designing a real-time 360° video streaming system to HMDs quite tricky, especially because most video streaming solutions nowadays deliver videos in segments lasting for a few seconds.

Figure 1: (a) 360° videos contain much more information than regular videos.
Figure 1: (b) viewer only sees a small part of the 360° video.

Goals and Overview

View details »