An Overview of OpenAI's Sora and Its Potential for Physics Engine Free Games and Virtual Reality

Zuyan Chen; Shuai Li; Md. Asraful Haque

doi:10.4108/airo.5273

Authors

Zuyan Chen Swansea University
Shuai Li University of Oulu
Md. Asraful Haque Aligarh Muslim University

DOI:

https://doi.org/10.4108/airo.5273

Keywords:

Sora, World Model, GPT, Physics Engine, Virtual Reality, Game, Simulator, Text to Video, Video Generation

Abstract

Sora, OpenAI's latest text-to-video model, is particularly skilled at understanding the physical world, and all of the content it generates mostly consistent with the laws of physics. This indicates that Sora already has the beginnings of a world model and has the potential to become an excellent physics engine in the near future. This paper analyses and explains in detail the potential applications of Sora in physics engines and virtual reality. In addition, its advantages and disadvantages over traditional physics engines are compared based on its unique behavioural characteristics. Finally, it looks forward to the application of Sora in other fields.

Downloads

Download data is not yet available.

References

Floridi, L. and Chiriatti, M. (2020) Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines 30: 681–694.

Anil, R., Dai, A.M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S. et al. (2023) Palm 2 technical report. arXiv preprint arXiv:2305.10403 .

Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y. and Narasimhan, K. (2024) Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems 36.

Haque, M.A. and Li, S. (2023) The potential use of chatgpt for debugging and bug fixing. EAI Endorsed Transactions on AI and Robotics 2(1): e4–e4.

Haque, M.A. and Li, S. (2024) Exploring chatgpt and its impact on society. AI and Ethics : 1–13.

Dhariwal, P. and Nichol, A. (2021) Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34: 8780–8794.

Rombach, R., Blattmann, A., Lorenz, D., Esser, P. and Ommer, B. (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition: 10684–10695.

Brooks, T., Peebles, B., Homes, C., DePue, W., Guo, Y., Jing, L., Schnurr, D. et al. (2024) Video generation models as world simulators URL https://openai.com/research/video-generation-models-as-world-simulators.

Ha, D. and Schmidhuber, J. (2018) World models. arXiv preprint arXiv:1803.10122.

Todorov, E., Erez, T. and Tassa, Y. (2012) Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems (IEEE): 5026–5033.

Degrave, J., Hermans, M., Dambre, J. et al. (2019) A differentiable physics engine for deep learning in robotics. Frontiers in neurorobotics : 6.

Haas, J.K. (2014) A history of the unity game engine. Diss. Worcester Polytechnic Institute 483(2014): 484.

Qiu, W. and Yuille, A. (2016) Unrealcv: Connecting computer vision to unreal engine. In Computer Vision– ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14 (Springer): 909–916.

Hummel, J., Wolff, R., Stein, T., Gerndt, A. and Kuhlen, T. (2012) An evaluation of open source physics engines for use in virtual reality assembly simulations. In Advances in Visual Computing: 8th International Symposium, ISVC 2012, Rethymnon, Crete, Greece, July 16-18, 2012, Revised Selected Papers, Part II 8 (Springer): 346–357.

Maciel, A., Halic, T., Lu, Z., Nedel, L.P. and De, S. (2009) Using the physx engine for physics-based virtual surgery with force feedback. The International Journal of Medical Robotics and Computer Assisted Surgery 5(3): 341– 353.

Hafner, D., Lillicrap, T., Norouzi, M. and Ba, J. (2020) Mastering atari with discrete world models. arXiv preprint arXiv:2010.02193 .

Peng, X.B., Andrychowicz, M., Zaremba, W. and Abbeel, P. (2018) Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA) (IEEE): 3803–3810.

Wu, J., Yildirim, I., Lim, J.J., Freeman, B. and Tenenbaum, J. (2015) Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. Advances in neural information processing systems 28.

Koenig, N. and Howard, A. (2004) Design and use paradigms for gazebo, an open-source multi-robot simulator. In 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS)(IEEE Cat. No. 04CH37566) (IEEE), 3: 2149–2154.