An Accurate Viewport Estimation Method for 360 Video Streaming using Deep Learning

Authors

DOI:

https://doi.org/10.4108/eetinis.v9i4.2218

Abstract

Nowadays, Virtual Reality is becoming more and more popular, and 360 video is a very important part of the system. 360 video transmission over the Internet faces many difficulties due to its large size. Therefore, to reduce the network bandwidth requirement of 360-degree video, Viewport Adaptive Streaming (VAS) was proposed. An important issue in VAS is how to estimate future user viewing direction. In this paper, we propose an algorithm called GLVP (GRU-LSTM-based-Viewport-Prediction) to estimate the typical view for the VAS system. The results show that our method can improve viewport estimation from 9.5% to near 20%compared with other methods.

Downloads

Download data is not yet available.

References

D. V. Nguyen, H. T. T. Tran, and T. C. Thang, “An evaluation of tile selection methods for viewport-adaptive streaming of 360-degree video,” ACM Trans. Multimedia Comput. Commun. Appl., vol. 16, no. 1, mar 2020. [Online]. Available: https://doi.org/10.1145/3373359

D. V. Nguyen, H. T. Tran, and T. C. Thang, “Impact of delays on 360-degree video communications,” in 2017 TRON Symposium (TRONSHOW). IEEE, 2017, pp. 1–6.

F. Qian, L. Ji, B. Han, and V. Gopalakrishnan, “Optimizing 360 video delivery over cellular networks,” in Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges, ser. ATC ’16. New York, NY, USA: Association for Computing [4] Machinery, 2016, p. 1–6. [Online]. Available: https://doi.org/10.1145/2980055.2980056

Y. Bao, H. Wu, T. Zhang, A. A. Ramli, and X. Liu, “Shooting a moving target: Motion-prediction-based transmission for 360-degree videos,” in 2016 IEEE International Conference on Big Data (Big Data), 2016, pp. 1161–1170.

C.-L. Fan, J. Lee, W.-C. Lo, C.-Y. Huang, K.-T. Chen, and C.-H. Hsu, “Fixation prediction for 360 video streaming in head-mounted virtual reality,” in Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video, 2017, pp. 67–72.

Y. Ban, L. Xie, Z. Xu, X. Zhang, Z. Guo, and Y. Wang, “: Exploiting cross-users behaviors for viewport prediction in 360 video adaptive streaming,” in 2018 IEEE International Conference on Multimedia and Expo (ICME), 2018, pp. 1–6.

C. Wu, R. Zhang, Z. Wang, and L. Sun, “A spherical convolution approach for learning long term viewport prediction in 360 immersive video,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, 2020, pp. 14 003–14 040.

M. Hosseini and V. Swaminathan, “Adaptive 360 vr video streaming: Divide and conquer,” in 2016 IEEE International Symposium on Multimedia (ISM), 2016, pp. 107–110.

D. V. Nguyen, H. T. T. Tran, A. T. Pham, and T. C. Thang, “A new adaptation approach for viewport-adaptive 360-degree video streaming,” in 2017 IEEE International Symposium on Multimedia (ISM), 2017, pp. 38–44.

A. Zare, A. Aminlou, M. M. Hannuksela, and M. Gabbouj, “Hevc-compliant tile-based streaming of panoramic video for virtual reality applications,” in Proceedings of the 24th ACM International Conference on Multimedia, ser. MM ’16. New York, NY, USA: Association for Computing Machinery, 2016, p. 601–605.[Online]. Available: https://doi.org/10.1145/2964284. 2967292

D. V. Nguyen, H. T. T. Tran, A. T. Pham, and T. C. Thang, “An optimal tile-based approach for viewport-adaptive 360-degree video streaming,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 1, pp. 29–42, 2019.

S. Petrangeli, V. Swaminathan, M. Hosseini, and

F. De Turck, “An http/2-based adaptive streaming framework for 360° virtual reality videos,” in Proceedings of the 25th ACM International Conference on Multimedia, ser. MM ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 306–314. [Online]. Available: https://doi.org/10.1145/3123266.3123453

Z. Xu, X. Zhang, K. Zhang, and Z. Guo, “Probabilistic viewport adaptive streaming for 360-degree videos,” in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), 2018, pp. 1–5.

C. Li, W. Zhang, Y. Liu, and Y. Wang, “Very long term field of view prediction for 360-degree video streaming,” in 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, 2019, pp. 297–302.

X. Feng, V. Swaminathan, and S. Wei, “Viewport prediction for live 360-degree mobile video streaming using user-content hybrid motion tracking,” Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 3, no. 2, jun 2019. [Online]. Available: https://doi.org/10.1145/3328914

J. Heyse, M. T. Vega, F. de Backere, and F. de Turck, “Contextual bandit learning-based viewport prediction for 360 video,” in 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019, pp. 972–973.

L. Sun, Y. Mao, T. Zong, Y. Liu, and Y. Wang, “Flocking-based live streaming of 360-degree video,” in Proceedings of the 11th ACM Multimedia Systems Conference, ser. MMSys ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 26–37. [Online]. Available: https://doi.org/10.1145/3339825.3391856

A. T. Nasrabadi, A. Samiei, and R. Prakash, “Viewport prediction for 360° videos: A clustering approach,” in Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, ser. NOSSDAV ’20. New York, NY, USA: Association for Computing Machinery, 2020, p. 34–39. [Online]. Available: https://doi.org/10.1145/3386290.3396934

J. Park, M. Wu, K.-Y. Lee, B. Chen, K. Nahrstedt, M. Zink, and R. Sitaraman, “Seaware: Semantic aware view prediction system for 360-degree video streaming,” in 2020 IEEE International Symposium on Multimedia (ISM), 2020, pp. 57–64.

T. C. Thang, H. T. Le, A. T. Pham, and Y. M. Ro, “An evaluation of bitrate adaptation methods for http live streaming,” IEEE Journal on Selected Areas in Communications, vol. 32, no. 4, pp. 693–705, April 2014.

X. Corbillon, F. De Simone, and G. Simon, “360-degree video head movement dataset,” in Proceedings of the 8th ACM on Multimedia Systems Conference, ser. MMSys’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 199–204. [Online]. Available: https://doi.org/10.1145/3083187.3083215

D. Nguyen, “An evaluation of viewport estimation methods in 360-degree video streaming,” in 2022 7th International Conference on Business and Industrial Research (ICBIR), 2022, pp. 161–166.

Downloads

Published

21-09-2022

How to Cite

Nguyen, H., Dao, T. N., Pham, N. S., Dang, T. L., Nguyen, T. D., & Truong, T. H. (2022). An Accurate Viewport Estimation Method for 360 Video Streaming using Deep Learning. EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, 9(4), e2. https://doi.org/10.4108/eetinis.v9i4.2218