A Hybrid Framework for Visual Positioning: Combining Convolutional Neural Networks with Ontologies





Visual Positioning, Convolutional Neural Network, Ontology, Image Understanding


Visual positioning is a new generation positioning technique which has been developed rapidly during recent years for many applications such as robotics, self-driving vehicles and positioning for visually impaired people due to advent of powerful image processing methods, especially Convolutional Neural Networks. Nowadays, deep Convolutional Neural Networks are capable of classifying images with high accuracy rates; however, comparing visual perception by a human being, pure Neural Networks lack background knowledge which is essential for estimating the position through a reasoning process. In this paper we present a hybrid framework for employing ontologies over Convolutional Neural Networks to integrate a knowledge-based reasoning with Neural Networks for taking advantages of capabilities similar to human brain’s functions. The proposed framework is generic so it can be applied to a wide variety of scenarios in smart cities where visual positioning represents added value.


Download data is not yet available.


Aguirre, E.; Garcia-Silvente, M.; Plata, J. Leg: Detection and Tracking for a Mobile Robot and Based on a Laser Device, Supervised Learning and Particle Filtering. In ROBOT2013: First Iberian Robotics Conference (2013).

Aulinas, J., Petillot, Y., Salvi, J., Lladó, X.: The SLAM problem: a survey. In Proceedings of the 2008 conference on Artificial Intelligence Research and Development, 363–371 (2008).

Breen C, Khan L, Kumar A.: Ontology-based image classification using neural networks. Proceedings of the ITCom 2002: The Convergence of Information Technologies and Communications (2002).

Burroughes, G.,Gao, Y.: Ontology-based self-reconfiguring guidance, navigation, and control for planetary rovers. Journal of Aerospace Information Systems 13, 316–328 (2016).

Chen, K., Wang, C., Wei, X.: Vision-Based Positioning for Internet-of-Vehicles, IEEE Transactions on Intelligent Transportation Systems 18 (2), pp. 364-376 (2016).

Deretey, E., Ahmed, M. T., Marshall, J. A., Greenspan, M.: Visual indoor positioning with a single camera using PnP, International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1-9, doi:10.1109/IPIN.2015. 7346756 (2015).

Fan, J., Fang, L., Wu, J.: From Brain Science to Artificial Intelligence, Engineering, Volume 6, Issue 3, Pages 248-252 (2020).

Goldman-Rakic P.S.: Cellular and circuit basis of working memory in prefrontal cortex of nonhuman primates, Prog Brain Res, 85, pp. 325-335 (1990).

Gupta, U., Chaudhury, S.: Deep transfer learning with ontology for image classification, Proceedings of the Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, 1-4 (2015).

Kendall, A., Grimes, Matthew, Cipolla, Roberto: Convolutional networks for real-time 6-DOF camera relocalization (2015).

Kostavelis, I., Gasteratos, A.: Semantic mapping for mobile robotics tasks, Robotics and Autonomous Systems, Volume 66(C), pp 86–103 (2015).

Li, B., Muñoz, JP., Rong, X., Chen, Q., Xiao J, Tian Y, Arditi A, Yousuf M: Vision-based Mobile Indoor Assistive Navigation Aid for Blind People. IEEE Trans Mob Computer, V18 (3):702-714 (2019).

Li, B.; Wu, T.; Shuai1, S.; Zhang, L.; Chu, R.: Object Detection via Aspect Ratio and Context Aware Region-based Convolutional Networks, arXiv: 1612.00534v2 (2017).

Liu, J., Feng-Ping, A.: Image Classification Algorithm Based on Deep Learning-Kernel Function Scientific Programming, 7607612 (2020).

Monroy, J.; Ruiz-Sarmiento, J.; Moreno, F.; Galindo, C.; Gonzalez-Jimenez, J. Olfaction: Vision, and Semantics for Mobile Robots. Results of the IRO Project. Sensors 19, 3488 (2019).

Rahman Su, Ullah S, Ullah S: A mobile camera based navigation system for visually impaired people. In: Proceedings of the 7th international conference on communications and broadband networking, pp 63–66 (2019).

Schill, K.,Zetzsche, C., Hois, J.: A belief-based architecture for scene analysis: From sensorimotor features to knowledge and ontology, Fuzzy Sets and Systems, V 160(10), Pages 1507-1516, (2009).

Studer, R., Benjamins, V., Fensel, R.: Knowledge engineering: principles and methods. Data & Knowledge Engineering. 25. 161-197 (1998).

Tsai S F.: Toward ontological visual understanding. Dissertations & Theses - Gradworks (2012).




How to Cite

Mosaddegh A, Lopes S, Rostami H, Keshavarz A, Paiva S. A Hybrid Framework for Visual Positioning: Combining Convolutional Neural Networks with Ontologies. EAI Endorsed Trans Energy Web [Internet]. 2022 Dec. 29 [cited 2023 Feb. 6];9(40):e6. Available from: https://publications.eai.eu/index.php/ew/article/view/2959