Cloud-Edge Orchestration for Smart Cities: A Review of Kubernetes-based Orchestration Architectures

Sebastian Böhm; Guido Wirtz

doi:10.4108/eetsc.v6i18.1197

Authors

Sebastian Böhm Distributed Systems Group, University of Bamberg, Bamberg, Germany
Guido Wirtz Distributed Systems Group, University of Bamberg, Bamberg, Germany

DOI:

https://doi.org/10.4108/eetsc.v6i18.1197

Keywords:

Edge computing, Edge orchestration, Cloud computing, Container orchestration, Kubernetes

Abstract

Edge computing offers computational resources near data-generating devices to enable low-latency access. Especially for smart city contexts, edge computing becomes inevitable for providing real-time services, like air quality monitoring systems. Kubernetes, a popular container orchestration platform, is often used to efficiently manage containerized applications in smart cities. Although it misses essential requirements of edge computing, like network-related metrics for scheduling decisions, it is still considered. This paper analyzes custom cloud-edge architectures implemented with Kubernetes. Specifically, we analyze how essential requirements of edge orchestration in smart cities are solved. Also, shortcomings are identified in these architectures based on the fundamental requirements of edge orchestration. We conduct a literature review to obtain the general requirements of edge computing and edge orchestration for our analysis. We map these requirements to the capabilities of Kubernetes-based cloud-edge architectures to assess their level of achievement. Issues like using network-related metrics and the missing topology-awareness of networks are partially solved. However, requirements like real-time resource utilization, fault-tolerance, and the placement of container registries are in the early stages. We conclude that Kubernetes is an eligible candidate for cloudedge orchestration. When the formerly mentioned issues are solved, Kubernetes can successfully contribute latency-critical, large-scale, and multi-tenant application deployments for smart cities.

Downloads

Download data is not yet available.

References

[1] Sebrechts, M., Borny, S.,Wauters, T., Volckaert, B. and Turck, F.D. (2021) Service relationship orchestration: Lessons learned from running large scale smart city platforms on kubernetes. IEEE Access 9: 133387–133401.

[2] Ramaprasad, A., Sánchez-Ortiz, A. and Syn, T. (2017) A unified definition of a smart city. In Lecture Notes in Computer Science, 13–24.

[3] Santos, J., Vanhove, T., Sebrechts, M., Dupont, T., Kerckhove,W., Braem, B., Seghbroeck, G.V. et al. (2018) City of things: Enabling resource provisioning in smart cities. IEEE Communications Magazine 56(7): 177–183.

[4] Su, K., Li, J. and Fu, H. (2011) Smart city and the applications. In 2011 International Conference on Electronics, Communications and Control (ICECC) (IEEE).

[5] Gaur, A., Scotney, B., Parr, G. and McClean, S. (2015) Smart city architecture and its applications based on IoT. Procedia Computer Science 52: 1089–1094.

[6] hoon Kim, T., Ramos, C. andMohammed, S. (2017) Smart city and IoT. Future Generation Computer Systems 76: 159–162. [7] Eremia, M., Toma, L. and Sanduleac, M. (2017) The smart city concept in the 21st century. Procedia Engineering 181: 12–19.

[8] Atzori, L., Iera, A. andMorabito, G. (2010) The internet of things: A survey. Computer Networks 54(15): 2787–2805.

[9] Mohamed, N., Al-Jaroodi, J. and Jawhar, I. (2019) Towards fault tolerant fog computing for IoT-based smart city applications. In 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC) (IEEE).

[10] Tcholtchev, N. and Schieferdecker, I. (2021) Sustainable and reliable information and communication technology for resilient smart cities. Smart Cities 4(1): 156–176.

[11] Kumar, V., Laghari, A.A., Karim, S., Shakir, M. and Brohi, A.A. (2019) Comparison of fog computing & cloud computing. Int. J. Math. Sci. Comput 1: 31–41.

[12] Lau, B.P.L., Marakkalage, S.H., Zhou, Y., Hassan, N.U., Yuen, C., Zhang, M. and Tan, U.X. (2019) A survey of data fusion in smart city applications. Information Fusion 52: 357–374.

[13] Shahrour, I. and Xie, X. (2021) Role of internet of things (IoT) and crowdsourcing in smart city projects. Smart Cities 4(4): 1276–1292.

[14] Balakrishna, C. (2012) Enabling technologies for smart city services and applications. In 2012 Sixth International Conference on Next Generation Mobile Applications, Services and Technologies (IEEE).

[15] Laghari, A.A., He, H., Shafiq, M. and Khan, A. (2016) Assessing effect of cloud distance on end user's quality of experience (QoE). In 2016 2nd IEEE International Conference on Computer and Communications (ICCC) (IEEE).

[16] Cao, J., Zhang, Q. and Shi, W. (2018) Edge Computing: A Primer (Springer International Publishing).

[17] Shi, W., Cao, J., Zhang, Q., Li, Y. and Xu, L. (2016) Edge computing: Vision and challenges. IEEE Internet of Things Journal 3(5): 637–646.

[18] Caragliu, A., Bo, C.D. and Nijkamp, P. (2011) Smart cities in europe. Journal of Urban Technology 18(2): 65–82.

[19] Satyanarayanan, M. (2017) Edge computing. Computer 50(10): 36–38.

[20] Laghari, A.A., He, H., Memon, K.A., Laghari, R.A., Halepoto, I.A. and Khan, A. (2019) Quality of experience (QoE) in cloud gaming models: A review. Multiagent and Grid Systems 15(3): 289–304.

[21] Ogbuachi, M.C., Reale, A., Suskovics, P. and Kovács, B. (2020) Context-aware kubernetes scheduler for edgenative applications on 5g. Journal of communications software and systems 16(1): 85–94.

[22] Morabito, R. (2017) Virtualization on internet of things edge devices with container technologies: A performance evaluation. IEEE Access 5: 8835–8850.

[23] Pahl, C. and Lee, B. (2015) Containers and clusters for edge cloud architectures – a technology review. In 2015 3rd International Conference on Future Internet of Things and Cloud (IEEE).

[24] Amaral, M., Polo, J., Carrera, D., Mohomed, I., Unuvar, M. and Steinder, M. (2015) Performance evaluation of microservices architectures using containers. In 2015 IEEE 14th International Symposium on Network Computing and Applications: 27–34.

[25] Barika, M., Garg, S., Zomaya, A.Y., Wang, L., Moorsel, A.V. and Ranjan, R. (2019) Orchestrating big data analysis workflows in the cloud. ACMComputing Surveys 52(5): 1–41.

[26] Santos, J., Wauters, T., Volckaert, B. and Turck, F.D. (2019) Resource provisioning in fog computing: From theory to practice †. Sensors 19(10): 2238.

[27] Barriga, J.J., Sulca, J., León, J., Ulloa, A., Portero, D., García, J. and Yoo, S.G. (2020) A smart parking solution architecture based on LoRaWAN and kubernetes. Applied Sciences 10(13): 4674.

[28] Ogawa, K., Kanai, K., Nakamura, K., Kanemitsu, H., Katto, J. and Nakazato, H. (2019) IoT device virtualization for efficient resource utilization in smart city IoT platform. In 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (IEEE).

[29] Kristiani, E., Yang, C.T., Huang, C.Y., Wang, Y.T. and Ko, P.C. (2020) The implementation of a cloudedge computing architecture using OpenStack and kubernetes for air quality monitoring application. Mobile Networks and Applications : 1–23.

[30] Albino, V., Berardi, U. and Dangelico, R.M. (2015) Smart cities: Definitions, dimensions, performance, and initiatives. Journal of Urban Technology 22(1): 3–21.

[31] Dameri, R. and Cocchia, A. (2013) Smart city and digital city: twenty years of terminology evolution. ITAIS : 1–8.

[32] Wenge, R., Zhang, X., Dave, C., Chao, L. and Hao, S. (2014) Smart city architecture: A technology guide for implementation and design challenges. China Communications 11(3): 56–69.

[33] Anthopoulos, L. (2015) Defining smart city architecture for sustainability.

[34] Yin, C., Xiong, Z., Chen, H., Wang, J., Cooper, D. and David, B. (2015) A literature survey on smart cities. Science China Information Sciences 58(10): 1–18.

[35] Al-Farabi, M., Chowdhury, M., Readuzzaman, M., Hossain, M., Sabuj, S. and Hossain, M. (2018) Smart environment monitoring system using unmanned aerial vehicle in bangladesh. EAI Endorsed Transactions on Smart Cities .

[36] Hollands, R.G. (2008) Will the real smart city please stand up? City 12(3): 303–320.

[37] Park, S., Park, S., Park, L., Park, S., Lee, S., Lee, T., Lee, S. et al. (2018) Design and implementation of a smart IoT based building and town disaster management system in smart city infrastructure. Applied Sciences 8(11): 2239.

[38] Sakhardande, P., Hanagal, S. and Kulkarni, S. (2016) Design of disaster management system using IoT based interconnected network with smart city monitoring. In 2016 International Conference on Internet of Things and Applications (IOTA) (IEEE).

[39] Tönjes, R., Barnaghi, P., Ali, M., Mileo, A., Hauswirth, M., Ganz, F., Ganea, S. et al. (2014) Real time iot stream processing and large-scale data analytics for smart city applications. In European Conference on Networks and Communications (sn): 10.

[40] Hafez, R. (2018) A methodical plan towards smart economy in new egyptian cities. EAI Endorsed Transactions on Smart Cities .

[41] Mulligan, C.E.A. and Olsson, M. (2013) Architectural implications of smart city business models: an evolutionary perspective. IEEE Communications Magazine 51(6): 80–85.

[42] Saharan, S., Kumar, N. and Bawa, S. (2020) An efficient smart parking pricing system for smart city environment: A machine-learning based approach. Future Generation Computer Systems 106: 622–640.

[43] Baker, N., Szabo-Müller, P. and Handmann, U. (2018) Transfer learning-based method for automated e-waste recycling in smart cities. EAI Endorsed Transactions on Smart Cities .

[44] Siregar, B., Nasution, A.B.A. and Fahmi, F. (2016) Integrated pollution monitoring system for smart city. In 2016 International Conference on ICT For Smart Society (ICISS) (IEEE).

[45] Honarvar, A.R. and Sami, A. (2019) Towards sustainable smart city by particulate matter prediction using urban big data, excluding expensive air pollution infrastructures. Big Data Research 17: 56–65.

[46] Bagchi, S., Siddiqui, M.B., Wood, P. and Zhang, H. (2019) Dependability in edge computing. Communications of the ACM 63(1): 58–66.

[47] Laghari, A.A., Jumani, A.K. and Laghari, R.A. (2021) Review and state of art of fog computing. Archives of Computational Methods in Engineering 28(5): 3631–3643.

[48] Svorobej, S., Bendechache, M., Griesinger, F. and Domaschka, J. (2020) Orchestration from the cloud to the edge. In The Cloud-to-Thing Continuum, 61–77.

[49] Laghari, A.A.,Wu, K., Laghari, R.A., Ali, M. and Khan, A.A. (2021) A review and state of art of internet of things (IoT). Archives of Computational Methods in Engineering 29(3): 1395–1413.

[50] Laghari, A., Laghari, R., Wagan, A. and Umrani, A. (2018) Effect of packet loss and reorder on quality of audio streaming. ICST Transactions on Scalable Information Systems .

[51] Premsankar, G., Francesco, M.D. and Taleb, T. (2018) Edge computing for the internet of things: A case study. IEEE Internet of Things Journal 5(2): 1275–1284.

[52] Bilal, K., Khalid, O., Erbad, A. and Khan, S.U. (2018) Potentials, trends, and prospects in edge technologies: Fog, cloudlet, mobile edge, and micro data centers. Computer Networks 130: 94–120.

[53] Ahmed, E. and Rehmani, M.H. (2017) Mobile edge computing: Opportunities, solutions, and challenges. Future Generation Computer Systems 70: 59–63.

[54] Satyanarayanan, M., Bahl, P., Caceres, R. and Davies, N. (2009) The case for VM-based cloudlets in mobile computing. IEEE Pervasive Computing 8(4): 14–23.

[55] Al-Tarawneh, M.A.B. (2020) Mobility-aware container migration in cloudlet-enabled IoT systems using integrated muticriteria decision making. International Journal of Advanced Computer Science and Applications 11(9).

[56] Qiu, Y., Lung, C.H., Ajila, S. and Srivastava, P. (2017) LXC container migration in cloudlets under multipath TCP. In 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC) (IEEE).

[57] Qiu, Y., Lung, C.H., Ajila, S. and Srivastava, P. (2019) Experimental evaluation of LXC container migration for cloudlets using multipath TCP. Computer Networks 164: 106900.

[58] Böhm, S. and Wirtz, G. (2021) Towards orchestration of cloud-edge architectures with kubernetes. EAI Edge-IoT 2021 - 2nd EAI International Conference on Intelligent Edge Processing in the IoT Era .

[59] Varghese, B., Wang, N., Barbhuiya, S., Kilpatrick, P. and Nikolopoulos, D.S. (2016) Challenges and opportunities in edge computing. In 2016 IEEE International Conference on Smart Cloud (SmartCloud): 20–26.

[60] Bonomi, F., Milito, R., Zhu, J. and Addepalli, S. (2012) Fog computing and its role in the internet of things. In Proceedings of the first edition of the MCC workshop on Mobile cloud computing - MCC '12 (ACM Press).

[61] Vaquero, L.M., Cuadrado, F., Elkhatib, Y., Bernal-Bernabe, J., Srirama, S.N. and Zhani, M.F. (2019) Research challenges in next gen service orchestration. Future Generation Computer Systems 90: 20–38.

[62] Aazam, M., Zeadally, S. and Harras, K.A. (2018) Offloading in fog computing for IoT: Review, enabling technologies, and research opportunities. Future Generation Computer Systems 87: 278–289.

[63] Hong, C.H. and Varghese, B. (2019) Resource management in fog/edge computing. ACM Computing Surveys 52(5): 1–37.

[64] da Silva, D.M.A., Asaamoning, G., Orrillo, H., Sofia, R.C. and Mendes, P.M. (2019) An analysis of fog computing data placement algorithms. In Proceedings of the 16th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (ACM).

[65] Velasquez, K., Abreu, D.P., Assis, M.R.M., Senna, C., Aranha, D.F., Bittencourt, L.F., Laranjeiro, N. et al. (2018) Fog orchestration for the internet of everything: state-of-the-art and research challenges. Journal of Internet Services and Applications 9(1).

[66] Pahl, C., Ioini, N.E., Helmer, S. and Lee, B. (2018) An architecture pattern for trusted orchestration in IoT edge clouds. In 2018 Third International Conference on Fog and Mobile Edge Computing (FMEC) (IEEE).

[67] Casalicchio, E. (2017) Autonomic orchestration of containers: Problem definition and research challenges. In Proceedings of the 10th EAI International Conference on Performance Evaluation Methodologies and Tools (ACM).

[68] Giordano, A., Spezzano, G. and Vinci, A. (2016) Smart agents and fog computing for smart city applications. In Smart Cities, 137–146.

[69] Cicirelli, F., Guerrieri, A., Spezzano, G. and Vinci, A. (2017) An edge-based platform for dynamic smart city applications. Future Generation Computer Systems 76: 106–118.

[70] Hsieh, Y.C., Hong, H.J., Tsai, P.H., Wang, Y.R., Zhu, Q., Uddin, M.Y.S., Venkatasubramanian, N. et al. (2018) Managed edge computing on internet-of-things devices for smart city applications. In NOMS 2018 - 2018 IEEE/IFIP Network Operations and Management Symposium (IEEE). [71] Barthélemy, J., Verstaevel, N., Forehead, H. and Perez, P. (2019) Edge-computing video analytics for real-time traffic monitoring in a smart city. Sensors 19(9): 2048.

[72] Hossain, S.A., Rahman, M.A. and Hossain, M.A. (2018) Edge computing framework for enabling situation awareness in IoT based smart city. Journal of Parallel and Distributed Computing 122: 226–237.

[73] Ahsaan, S. and Mourya, A. (2018) Prognostic modelling for smart cities using smart agents and IoT: A proposed solution for sustainable development. EAI Endorsed Transactions on Smart Cities .

[74] Taleb, T., Dutta, S., Ksentini, A., Iqbal, M. and Flinck, H. (2017) Mobile edge computing potential in making cities smarter. IEEE Communications Magazine 55(3): 38–43.

[75] Al-gaashani, M., Muthanna, M.S.A., Abdukodir, K., Muthanna, A. and Kirichek, R. (2020) Intelligent system architecture for smart city and its applications based edge computing. In 2020 12th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT) (IEEE).

[76] Hussain, M.M., Alam, M.S. and Beg, M.S. (2018) Computational viability of fog methodologies in IoT enabled smart city architectures-a smart grid case study. EAI Endorsed Transactions on Smart Cities .

[77] Gheisari, M., Pham, Q.V., Alazab, M., Zhang, X., Fernandez-Campusano, C. and Srivastava, G. (2019) ECA: An edge computing architecture for privacypreserving in IoT-based smart city. IEEE Access 7: 155779–155786.

[78] Deng, Y., Chen, Z., Yao, X., Hassan, S. and Wu, J. (2019) Task scheduling for smart city applications based on multi-server mobile edge computing. IEEE Access 7: 14410–14421.

[79] Zheng, X., Li, M. and Guo, J. (2020) Task scheduling using edge computing system in smart city. International Journal of Communication Systems 34(6).

[80] Wang, D., Bai, B., Lei, K., Zhao, W., Yang, Y. and Han, Z. (2019) Enhancing information security via physical layer approaches in heterogeneous IoT with multiple access mobile edge computing in smart city. IEEE Access 7: 54508–54521.

[81] Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Jalali, F., Niakanlahiji, A., Kong, J. et al. (2019) All one needs to know about fog computing and related edge computing paradigms: A complete survey. Journal of Systems Architecture 98: 289–330.

[82] Naha, R.K., Garg, S., Georgakopoulos, D., Jayaraman, P.P., Gao, L., Xiang, Y. and Ranjan, R. (2018) Fog computing: Survey of trends, architectures, requirements, and research directions. IEEE Access 6: 47980–48009.

[83] Hoque, S., Brito, M.S.D., Willner, A., Keil, O. and Magedanz, T. (2017) Towards container orchestration in fog computing infrastructures. In 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC) (IEEE).

[84] Mouradian, C., Naboulsi, D., Yangui, S., Glitho, R.H., Morrow, M.J. and Polakos, P.A. (2018) A comprehensive survey on fog computing: State-of-the-art and research challenges. IEEE Communications Surveys & Tutorials 20(1): 416–464.

[85] Wang, J., Pan, J., Esposito, F., Calyam, P., Yang, Z. and Mohapatra, P. (2019) Edge cloud offloading algorithms. ACM Computing Surveys 52(1): 1–23.

[86] Han, Y., Shen, S., Wang, X., Wang, S. and Leung, V.C.M. (2021) Tailored learning-based scheduling for kubernetes-oriented edge-cloud system. CoRR .

[87] Kaur, K., Garg, S., Kaddoum, G., Ahmed, S.H. and Atiquzzaman, M. (2020) KEIDS: Kubernetes-based energy and interference driven scheduler for industrial IoT in edge-cloud ecosystem. IEEE Internet of Things Journal 7(5): 4228–4237.

[88] Goethals, T., Volckaert, B. and de Turck, F. (2020) Adaptive fog service placement for real-time topology changes in kubernetes clusters. In Proceedings of the 10th International Conference on Cloud Computing and Services Science.

[89] Haja, D., Szalay, M., Sonkoly, B., Pongracz, G. and Toka, L. (2019) Sharpening kubernetes for the edge. In Proceedings of the ACM SIGCOMM 2019 Conference Posters and Demos on - SIGCOMM Posters and Demos '19 (ACM Press).

[90] Javed, A., Heljanko, K., Buda, A. and Framling, K. (2018) CEFIoT: A fault-tolerant IoT architecture for edge and cloud. In 2018 IEEE 4th World Forum on Internet of Things (WF-IoT) (IEEE): 813–818.

[91] Wöbker, C., Seitz, A., Mueller, H. and Bruegge, B. (2018) Fogernetes: Deployment and management of fog computing applications. In NOMS 2018 - 2018 IEEE/IFIP Network Operations and Management Symposium (IEEE).

[92] Kayal, P. (2020) Kubernetes in fog computing: Feasibility demonstration, limitations and improvement scope : Invited paper. In 2020 IEEE 6th World Forum on Internet of Things (WF-IoT) (IEEE): 1–6.

[93] Eidenbenz, R., Pignolet, Y.A. and Ryser, A. (2020) Latency-aware industrial fog application orchestration with kubernetes. In 2020 Fifth International Conference on Fog and Mobile Edge Computing (FMEC) (IEEE).

[94] Casquero, O., Armentia, A., Sarachaga, I., Perez, F., Orive, D. and Marcos, M. (2019) Distributed scheduling in kubernetes based on MAS for fog-in-the-loop applications. In 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA) (IEEE).

[95] Goethals, T., DeTurck, F. and Volckaert, B. (2020) Extending kubernetes clusters to low-resource edge devices using virtual kubelets. IEEE Transactions on Cloud Computing : 1–1.

[96] Böhm, S. and Wirtz, G. (2021) Profiling lightweight container platforms: Microk8s and k3s in comparison to kubernetes. In CEUR workshop proceedings (RWTH Aachen): 65–73.

[97] Taherizadeh, S., Stankovski, V. and Grobelnik, M. (2018) A capillary computing architecture for dynamic internet of things: Orchestration of microservices from edge devices to fog and cloud providers. Sensors 18(9): 2938.

[98] Yu, Z., Wang, J., Qi, Q., Liao, J. and Xu, J. (2018) Boundless application and resource based on container technology. In Edge Computing – EDGE 2018, 34–48.