A reawakening of Machine Learning Application in Unmanned Aerial Vehicle: Future Research Motivation
DOI:
https://doi.org/10.4108/eetiot.v8i29.987Keywords:
Machine learning, Reinforcement learning, supervised learning, non-supervised learning, semi-supervised learningAbstract
Machine learning (ML) entails artificial procedures that improve robotically through experience and using data. Supervised, unsupervised, semi-supervised, and Reinforcement Learning (RL) are the main types of ML. This study mainly focuses on RL and Deep learning, since necessitates mainly sequential and consecutive decision-making context. This is a comparison to supervised and non-supervised learning due to the interactive nature of the environment. Exploiting a forthcoming accumulative compensation and its stimulus of machines, complex policy decisions. The study further analyses and presents ML perspectives depicting state-of-the-art developments with advancement, relatively depicting the future trend of RL based on its applicability in technology. It's a challenge to an Internet of Things (IoT) and demonstrates what possibly can be adopted as a solution. This study presented a summarized perspective on identified arenas on the analysis of RL. The study scrutinized that a reasonable number of the techniques engrossed in alternating policy values instead of modifying other gears in an exact state of intellectual. The study presented a strong foundation for the current studies to be adopted by the researchers from different research backgrounds to develop models, and architectures that are relevant.
Downloads
References
W. Shafik, S. M. Matinkhah and M. Ghasemzadeh, "Theoretical understanding of deep learning in uav biomedical engineering technologies analysis," SN Computer Science, vol. 1, no. 6, pp. 1–13, 2020.
M. Azrour, J. Mabrouki, G. Fattah, A. Guezzaz, F. Aziz, “Machine learning algorithms for efficient water quality prediction,” Modeling Earth Systems and Environment, vol. 26, pp. 1-9, 2021. DOI: https://doi.org/10.1007/s40808-021-01266-6
M. Azrour, Y. Farhaoui, M. Ouanan and A. Guezzaz, “SPIT detection in telephony over IP using K-means algorithm,” Procedia computer science, vol. 148, pp. 542-51, 2019. DOI: https://doi.org/10.1016/j.procs.2019.01.027
A. Bu-Suhail, A. J. Al-Hulaibi, Z. Al-Khalaf, A. N. Jebril and Q. A. Al-Haija, “A comprehensive model driven secure mobile application for kfu email system,” Journal of Computer & Robotics, vol. 12, no. 1, pp. 25–38, 2019.
E. Vicente, J. Núñez‐Alfonsel, B. Ielpo, V. Ferri, R. Caruso et. al., "A cost‐effectiveness analysis of robotic versus laparoscopic distal pancreatectomy," International Journal of Medical Robotics and Computer Assisted Surgery, vol. 16, no. 2, 2020. DOI: https://doi.org/10.1002/rcs.2080
H. Meng, W. Shafik, S. M. Matinkhah and Z. Ahmad, "A 5g beam selection machine learning algorithm for unmanned aerial vehicle applications," Wireless Communications and Mobile Computing, 2020. DOI: https://doi.org/10.1155/2020/1428968
W. Shafik and S. A. Mostafavi, "Knowledge engineering on IoTs through reinforcement learning," International Journal of Computer Applications, vol.177, no. 44, pp. 0975–8887, 2019. DOI: https://doi.org/10.5120/ijca2020919952
W. Shafik, S. M. Matinkhah, M. Asadi, Z. Ahmadi and Z. Hadiyan, "A study on IoTs performance evaluation," Journal of Communications Technology, Electronics and Computer Science, vol. 28, pp. 1–19, 2020. DOI: https://doi.org/10.1007/s42979-020-00323-8
A. Guezzaz, S. Benkirane, M. Azrour, S. Khurram, “A Reliable Network Intrusion Detection Approach Using Decision Tree with Enhanced Data Quality,” Security and Communication Networks, vol. 21, 2021. DOI: https://doi.org/10.1155/2021/1230593
G. Tuysuzoglu and D. Birant, “Enhanced bagging (eBagging): A novel approach for ensemble learning,” Int. Arab. J. Inf. Technol, vol. 17, no. 4, pp. 515-528, 2020. DOI: https://doi.org/10.34028/iajit/17/4/10
B. Xu, X. Hu, X. Tang, X. Lin, H. Li et al., “Ensemble reinforcement learning-based supervisory control of hybrid electric vehicle for fuel economy improvement,” IEEE Transactions on Transportation Electrification, vol. 6, no. 2, pp. 717-727, 2020. DOI: https://doi.org/10.1109/TTE.2020.2991079
S. M. Matinkhah and W. Shafik, "A study on financial pricing and applications models on 5g," 4th Conference on Financial Mathematics and Application, Yazd, Iran, pp. 54–60, 2019.
S. M. Matinkhah and W. Shafik, "Broadcast communication analysis for 5g media radio access networks," 16th Conference on Broadcast and Exhibition, Tehran, Iran, 2019.
W. Shafik and S. M. Matinkhah, "Admitting new requests in fog networks according to erlang b distribution," IEEE 27th Iranian Conference on Electrical Engineering, Yazd, Iran, pp. 1–6, 2019. DOI: https://doi.org/10.1109/IranianCEE.2019.8786518
S. Mostafavi and W. Shafik, "Fog computing architectures, privacy and security solutions," Journal of Communications Technology, Electronics and Computer Science, vol. 24, pp. 1–14, 2019.
S. M. Matinkhah and W. Shafik, "Smart grid empowered by 5g technology," IEEE. Smart Grid Conference, Tehran, Iran, pp. 1–6, 2019. DOI: https://doi.org/10.1109/SGC49328.2019.9056590
W. Shafik, S. M. Matinkhah and M. Ghasemazade, "Fog-mobile edge performance evaluation and analysis on IoTs," Journal of Advance Research in Mobile Computing, vol. 1, no. 3, pp. 1–17, 2019.
W. Shafik and S. M. Matinkhah, "How to use erlang b to determine the blocking probability of packet loss in a wireless communication," 13th Symposium on Advances in Science & Technology, Mashhad, Iran, pp. 32124, 2018.
W. Shafik, S. M. Matinkhah and M. Ghasemzadeh, "A fast machine learning for 5g beam selection for unmanned aerial vehicle applications," Information Systems & Telecommunication, vol. 7, no. 28, pp. 262-278, 2019.
S. M. Matinkhah, W. Shafik and M. Ghasemzadeh, "Emerging artificial intelligence application: reinforcement learning issues on current IoTs," IEEE 16th International Conference in Information Knowledge and Technology, Tehran, Iran, 2019.
Q. Ma, E. Kobayashi, B. Fan, K. Nakagawa, I. Sakuma et al., "Automatic 3d landmarking model using patch‐based deep neural networks for ct image of oral and maxillofacial surgery," International Journal of Medical Robotics and Computer Assisted Surgery, vol. 16, no. 3, pp. e2093, 2020. DOI: https://doi.org/10.1002/rcs.2093
J. Dilley, H. Singh, P. Pratt, I. Omar, A. Darzi et al, "Visual behaviour in robotic surgery—demonstrating the validity of the simulated environment," International Journal of Medical Robotics and Computer Assisted Surgery, vol. 16, no. 2, 2020. DOI: https://doi.org/10.1002/rcs.2075
S. Veselic, G. Jocham, C. Gausterer, B. Wagner, M. Ressler et al., "A causal role for estradiol in human reinforcement learning," bioRxiv, 2020. DOI: https://doi.org/10.1101/2020.02.18.954982
W. Dabney, Z. Kurth-Nelson, N. Uchida, C. K. Starkweather, D. Hassabis et al, "A distributional code for value in dopamine-based reinforcement learning," Nature, vol. 577, no. 7792, pp. 671–675, 2020. DOI: https://doi.org/10.1038/s41586-019-1924-6
X. Wang, J. Cheng and L. Wang, "A reinforcement learning-based predator-prey model," Ecological Complexity, vol. 42, pp. 100815, 2020. DOI: https://doi.org/10.1016/j.ecocom.2020.100815
K. L. Tan, Y. Esfandiari, X. Y. Lee and S. Sarkar, "Robustifying reinforcement learning agents via action space adversarial training," IEEE American Control Conference, Denver, CO, USA, pp. 3959-3964, 2020. DOI: https://doi.org/10.23919/ACC45564.2020.9147846
K. Stephen and W. Banzhaf, "Temporal memory sharing in visual reinforcement learning," Genetic Programming Theory and Practice XVII, Springer, Cham, pp. 101-119. 2020. DOI: https://doi.org/10.1007/978-3-030-39958-0_6
C. E. Thornton, R. M. Buehrer, A. F. Martone and K. D. Sherbondy, "Experimental analysis of reinforcement learning techniques for spectrum sharing radar," IEEE International Radar Conference, Washington D.C., USA, pp. 67–72, 2020. DOI: https://doi.org/10.1109/RADAR42522.2020.9114698
A. Guezzaz, A. Asimi, Y. Asimi, M. Azrour and S. Benkirane, “A distributed intrusion detection approach based on machine leaning techniques for a cloud security,” In Intelligent Systems in Big Data, Semantic Web and Machine Learning, pp. 85-94, 2021. Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-72588-4_6
A. Ray, C. Chowdhury, S. Mallick, S. Mondal, S. Paul et al., "Designing Energy Efficient Strategies Using Markov Decision Process for Crowd-Sensing Applications," Mobile Networks and Applications, pp. 1–11, 2020. DOI: https://doi.org/10.1007/s11036-020-01522-6
W. Tao, N. Wang, B. Ellingwood and C. Nicholson, "Enhancing bridge performance following earthquakes using Markov decision process," Structure and Infrastructure Engineering, pp. 1-12, 2020. DOI: https://doi.org/10.1080/15732479.2020.1730410
M. Otten, J. Timmer and A. Witteveen, "Stratified breast cancer follow-up using a continuous state partially observable Markov decision process," European Journal of Operational Research, vol. 281, no. 2, pp. 464–474, 2020. DOI: https://doi.org/10.1016/j.ejor.2019.08.046
M. Azrour, J. Mabrouki and R. Chaganti, “New efficient and secured authentication protocol for remote healthcare systems in cloud-iot,” Security and Communication Networks, 2021. DOI: https://doi.org/10.1155/2021/5546334
Y. Zhang, R. Gan, J. Shao, H. Zhang and Y. Cheng, "Path selection with nash q-learning for remote state estimation over multihop relay network," International Journal of Robust and Nonlinear Control, vol. 30, no. 11, pp. 4331–4344, 2020. DOI: https://doi.org/10.1002/rnc.4945
D. Ding, X. Fan, Y. Zhao, K. Kang, Q. Yin, et al., "Q-learning based dynamic task scheduling for energy-efficient cloud computing," Future Generation Computer Systems, vol. 108, pp. 361–371, 2020. DOI: https://doi.org/10.1016/j.future.2020.02.018
W. Shafik, S. M. Matinkhah and M. Ghasemzadeh “Internet of things-based energy management, challenges, and solutions in smart cities,” Journal of Communications Technology, Electronics and Computer Science, vol. 27, 2020.
H. Mosadegh, S. F. Ghomi and G. A. Süer, "Stochastic mixed-model assembly line sequencing problem: mathematical modeling and q-learning based simulated annealing hyper-heuristics," European Journal of Operational Research, vol. 282, no. 2, pp 530–544, 2020. DOI: https://doi.org/10.1016/j.ejor.2019.09.021
R. Vafashoar and M. R. Meybodi, "A multi-population differential evolution algorithm based on cellular learning automata and evolutionary context information for optimization in dynamic environments," Applied Soft Computing, vol. 88, pp. 106009, 2020. DOI: https://doi.org/10.1016/j.asoc.2019.106009
L. Zhao, D. Zhu, W. Shafik, S. M. Matinkhah, Z. Ahmad et al., “Artificial intelligence analysis in cyber domain: A review,” International Journal of Distributed Sensor Networks, vol. 18, no. 4, 2022. DOI: https://doi.org/10.1177/15501329221084882
W. Shafik, "A fast machine learning for beam selection in 5g unmanned aerial vehicle communications" M.S. dissertation, Yazd University, Iran, 2020.
Z. Yang, L. Jianjun, H. Faqiri, W. Shafik, A. Talal Abdulrahman et al., “Green internet of things and big data application in smart cities development,” Complexity, 2021. DOI: https://doi.org/10.1155/2021/4922697
R. Fournier, L. Wang, O. V. Yazyev and Q. Wu, "Artificial neural network approach to the analytic continuation problem," Physical Review Letters, vol. 124, no. 5, pp. 056401, 2020. DOI: https://doi.org/10.1103/PhysRevLett.124.056401
W. Shafik, S. M. Matinkhah, “Unmanned aerial vehicles analysis to social networks performance,” The CSI Journal on Computer Science and Engineering, vol.18, no. 2, pp. 24-31, 2021.
L. Y. Leong, T. S. Hew, K. B. Ooi and J. Wei, "Predicting mobile wallet resistance: a two-staged structural equation modeling-artificial neural network approach," International Journal of Information Management, vol. 51, pp. 102047, 2020. DOI: https://doi.org/10.1016/j.ijinfomgt.2019.102047
Y. Lin, Z. Ahmad, W. Shafik, S. K. Khosa, Z. Almaspoor et al., “Impact of facebook and newspaper advertising on sales: a comparative study of online and print media,” Computational intelligence and neuroscience, 2021. DOI: https://doi.org/10.1155/2021/5995008
C. Wang, J. Wang, J. Wang and X. Zhang, "Deep reinforcement learning-based autonomous uav navigation with sparse rewards," IEEE IoTs Journal, vol. 7, no. 7, pp. 6180 – 6190, 2020.
W. Shafik, S. A. Mostafavi, “Knowledge engineering on internet of things through reinforcement learning,” Int. J. Comput. Appl, vol. 975, 2019.
R. Daniel, A. Radulescu and Y. Niv, "Intact reinforcement learning but impaired attentional control during multidimensional probabilistic learning in older adults," Journal of Neuroscience, vol. 40, no. 5, pp. 1084–1096, 2020. DOI: https://doi.org/10.1523/JNEUROSCI.0254-19.2019
S. M. Matinkhah and W. Shafik, “Broadcast Communication Analysis for 5G Media Radio Access Networks,” 16th Conference on Broadcast and Exhibition, Tehran, Iran, 2019.
P. Sun, Z. Guo, G. Wang, J. Lan and Y. Hu, "MARVEL: Enabling controller load balancing in software-defined networks with multi-agent reinforcement learning," Computer Networks, vol. 177, pp. 107230, 2020. DOI: https://doi.org/10.1016/j.comnet.2020.107230
C. H. Pi, K. C. Hu, S. Cheng and I. C. Wu, "Low-level autonomous control and tracking of quadrotor using reinforcement learning," Control Engineering Practice, vol. 95, pp. 104222, 2020. DOI: https://doi.org/10.1016/j.conengprac.2019.104222
B. Gu, X. Zhang, Z. Lin and M. Alazab, "Deep multi-agent reinforcement learning-based resource allocation for internet of controllable things," IEEE IoTs Journal, 2020. DOI: https://doi.org/10.1109/JIOT.2020.3023111
M. Yang, W. Huang, W. Tu, Q. Qu, and K. Lei, "Multitask learning and reinforcement learning for personalized dialog generation: an empirical study," IEEE Transactions on Neural Networks and Learning Systems, pp. 1–14, 2020. DOI: https://doi.org/10.1109/TNNLS.2020.2975035
N. Sendhilnathan, A. E. Ipata and M. E. Goldberg, "Neural correlates of reinforcement learning in mid-lateral cerebellum," Neuron, vol. 106, no. 1, pp.188-198, 2020. DOI: https://doi.org/10.1016/j.neuron.2019.12.032
O. Houidi, O. Soualah, W. Louati and D. Zeghlache, "An enhanced reinforcement learning approach for dynamic placement of virtual network functions," IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, pp. 1–7, 2020. DOI: https://doi.org/10.1109/PIMRC48278.2020.9217250
L. M. Hackel, P. Mende-Siedlecki and D. M. Amodio, "Reinforcement learning in social interaction: the distinguishing role of trait inference," Journal of Experimental Social Psychology, vol. 88, pp. 103948, 2020. DOI: https://doi.org/10.1016/j.jesp.2019.103948
D. J. Winkel, T. J. Weikert, H. C. Breit, G. Chabin, E. Gibson et al., "Validation of a fully automated liver segmentation algorithm using multi-scale deep reinforcement learning and comparison versus manual segmentation" European Journal of Radiology, pp. 108918, 2020. DOI: https://doi.org/10.1016/j.ejrad.2020.108918
D. Saba, Y. Sahli and A. Hadidi, "The role of artificial intelligence in company's decision making," Enabling AI Applications in Data Science, Springer, Cham, pp. 287–314. 2019. DOI: https://doi.org/10.1007/978-3-030-52067-0_13
W. Shafik, S. M. Matinkhah, M. N. Sanda and S. S. Afolabi, "A 3-dimensional fast machine learning algorithm for mobile unmanned aerial vehicle base stations," International Journal of Advances in Applied Sciences, vol. 10, no. 1, pp. 28–38, 2020. DOI: https://doi.org/10.11591/ijaas.v10.i1.pp28-38
W. Shafik, S. M. Matinkhah and M. N. Sanda, "Network resource management drives machine learning: a survey and future research direction," Journal of Communications Technology, Electronics and Computer Science, vol. 30, pp. 1–15, 2020.
W. Shafik, S. M. Matinkhah, P. Etemadinejad and M. N. Sanda, "Reinforcement learning rebirth, techniques, challenges, and resolutions," JOIV: International Journal on Informatics Visualization, vol. 4, no. 3, pp. 127–135, 2020. DOI: https://doi.org/10.30630/joiv.4.3.376
K. G. Dizaji, W. Chen and H. Huang, "Deep large-scale multi-task learning network for gene expression inference," International Conference on Research in Computational Molecular Biology, Springer, Cham, pp. 19–36. 2020. DOI: https://doi.org/10.1007/978-3-030-45257-5_2
T. H. Vu, A. Dang and J. C. Wang, "Learning to remember beauty products," in Proc. 28th ACM International Conference on Multimedia, pp. 4728-4732, 2020.
Y. Katsumi, A. M. Racine, A. Torrado-Carvajal, M. L. Loggia, J. M. Hooker et al., "The role of inflammation after surgery for elders (RISE) study: examination of [11c] PBR28 binding and exploration of its link to post-operative delirium" NeuroImage: Clinical, vol. 27, pp. 102346, 2020. DOI: https://doi.org/10.1016/j.nicl.2020.102346
W. Zheng, X. Zhu, G. Wen, Y. Zhu, H. Yu et al., "Unsupervised feature selection by self-paced learning regularization," Pattern Recognition Letters, vol. 132, pp. 4-11, 2020. DOI: https://doi.org/10.1016/j.patrec.2018.06.029
X. Zhang, M. Fan, D. Wang, P. Zhou and D. Tao, "Top-k feature selection framework using robust 0-1 integer programming," IEEE Transactions on Neural Networks and Learning Systems, pp. 1-15, 2020. DOI: https://doi.org/10.1109/TNNLS.2020.3009209
G. Kou, P. Yang, Y. Peng, F. Xiao, Y. Chen et al., "Evaluation of feature selection methods for text classification with small datasets using multiple criteria decision-making methods," Applied Soft Computing, vol. 86, pp. 105836, 2020. DOI: https://doi.org/10.1016/j.asoc.2019.105836
M. Mafarja, A. Qasem, A. A. Heidari, I. Aljarah, H. Faris et al., "Efficient hybrid nature-inspired binary optimizers for feature selection," Cognitive Computation, vol. 12, no. 1, pp. 150-175, 2020. DOI: https://doi.org/10.1007/s12559-019-09668-6
A. H. Williams, B. Poole, N. Maheswaranathan, A. K. Dhawale, T. Fisher et. at., "Discovering precise temporal patterns in large-scale neural recordings through robust and interpretable time warping," Neuron, vol. 105, no. 2, pp. 246-259, 2020. DOI: https://doi.org/10.1016/j.neuron.2019.10.020
M. Ghislieri, V. Agostini and M. Knaflitz, "Muscle synergies extracted using principal activations: improvement of robustness and interpretability," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 2, pp. 453-460, 2020. DOI: https://doi.org/10.1109/TNSRE.2020.2965179
G. Pesciullesi, P. Schwaller, T. Laino and J. L. Reymond, "Transfer learning enables the molecular transformer to predict regio-and stereoselective reactions on carbohydrates," Nature Communications, vol. 11, no. 1, pp. 1-8, 2020. DOI: https://doi.org/10.1038/s41467-020-18671-7
T. Lesort, V. Lomonaco, A. Stoian, D. Maltoni, D. Filliat et al., "Continual learning for robotics: definition, framework, learning strategies, opportunities and challenges," Information Fusion, vol. 58, pp. 52-68, 2020. DOI: https://doi.org/10.1016/j.inffus.2019.12.004
R. J. Collie, H. Granziera, A. J. Martin, E. C. Burns and A. J. Holliman, "Adaptability among science teachers in schools: A multi-nation examination of its role in school outcomes," Teaching and Teacher Education, vol. 95, pp. 103148, 2020. DOI: https://doi.org/10.1016/j.tate.2020.103148
S. M. Matinkhah and W. Shafik, “A Study on Financial Pricing and Applications models on 5G,” 4th Conference on Financial Mathematics and Applications, Yazd, Iran, 2019.
J. C. Cheng, W. Chen, K. Chen and Q. Wang, "Data-driven predictive maintenance planning framework for mep components based on bim and iot using machine learning algorithms," Automation in Construction, vol. 112, pp. 103087, 2020. DOI: https://doi.org/10.1016/j.autcon.2020.103087
Z. Tang, Y. Ye, Z. Jiang, C. Fu, R. Huang, et al., "A data-informed analytical approach to human-scale greenway planning: integrating multi-sourced urban data with machine learning algorithms, "Urban Forestry & Urban Greening, vol. 126871, 2020. DOI: https://doi.org/10.1016/j.ufug.2020.126871
M. F. Abdelwahed, A. E. Mohamed and M. A. Saleh, "Solving the motion planning problem using learning experience through case-based reasoning and machine learning algorithms," Ain Shams Engineering Journal, vol. 11, no. 1, pp. 133-142, 2020. DOI: https://doi.org/10.1016/j.asej.2019.10.007
N. Dupin and R. G. Talbi, "Machine learning-guided dual heuristics and new lower bounds for the refueling and maintenance planning problem of nuclear power plants," Algorithms, vol. 13, no. 8, pp. 185, 2020. DOI: https://doi.org/10.3390/a13080185
X. Xu, Y. Zhou, and Y. Leng, "Machine learning based image processing technology application in bunch longitudinal phase information extraction," Physical Review Accelerators and Beams, vol. 23, no. 3, pp. 032805, 2020. DOI: https://doi.org/10.1103/PhysRevAccelBeams.23.032805
E. Van 'T Wout, C. Pieringer, D. Torres Irribarra, K. Asahi and P. Larroulet, "Machine learning for policing: a case study on arrests in Chile," Policing and Society, pp. 1-15, 2020. DOI: https://doi.org/10.1080/10439463.2020.1779270
W. Shafik, S. M. Matinkhah and M. Ghasemzadeh, “A Mobile Fuzzy Sink Scheme for Wireless Sensor Network Period Improvement,” 18th IEEE Iranian Joint Congress on Fuzzy and intelligent Systems, pp. 11-216, 2020. DOI: https://doi.org/10.20944/preprints202007.0659.v1
K. Li, Y. Emami, W. Ni, E. Tovar, and Z. Han, “Onboard deep deterministic policy gradients for online flight resource allocation of uavs,” IEEE Networking Letters, vol. 2, no. 3, pp. 106–110, 2020. DOI: https://doi.org/10.1109/LNET.2020.3002341
M. S. Keszler, K. S. Wright, A. Miranda and M. S. Hopkins, "Multidisciplinary amputation team management of individuals with limb loss," Current Physical Medicine and Rehabilitation Reports, pp. 1-9, 2020. DOI: https://doi.org/10.1007/s40141-020-00282-4
F. Wu, H. Zhang, J. Wu, and L. Song, “Cellular UAV-to-Device Communications: Trajectory Design and Mode Selection by Multi-Agent Deep Reinforcement Learning,” IEEE Transactions on Communications, vol. 68, no. 7, pp. 4175–4189, 2020. DOI: https://doi.org/10.1109/TCOMM.2020.2986289
C. Wang, J. Wang, J. Wang, and X. Zhang, “Deep-reinforcement-learning-based autonomous uav navigation with sparse rewards,” IEEE IoTs Journal, vol. 7, no. 7, pp. 6180–6190, 2020. DOI: https://doi.org/10.1109/JIOT.2020.2973193
M. Mukherjee, V. Kumar, A. Lat, M. Guo, R. Matam, and Y. Lv, “Distributed Deep Learning-based Task Offloading for UAV-enabled Mobile Edge Computing,” in IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops, Jul. 2020, pp. 1208–1212, 2020. DOI: https://doi.org/10.1109/INFOCOMWKSHPS50562.2020.9162899
C. H. Liu, C. Piao, and J. Tang, “Energy-efficient uav crowdsensing with multiple charging stations by deep learning,” IEEE Conference on Computer Communications, pp. 199–208, 2020. DOI: https://doi.org/10.1109/INFOCOM41043.2020.9155535
L. B. Das et al., “Human Target Search and Detection using Autonomous UAV and Deep learning,” IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology, pp. 55–612020.
N. Zhao, Y. Cheng, Y. Pei, Y. Liang, and D. Niyato, “Deep Reinforcement Learning for Trajectory Design and Power Allocation in UAV Networks,” IEEE International Conference on Communications, pp. 1–6, 2020. DOI: https://doi.org/10.1109/ICC40277.2020.9149196
P. Tong, J. Liu, X. Wang, B. Bai, and H. Dai, “Deep Reinforcement Learning for Efficient Data Collection in UAV-Aided IoTs,” IEEE International Conference on Communications Workshops, pp. 1–6, 2020. DOI: https://doi.org/10.1109/ICCWorkshops49005.2020.9145249
C. H. Liu, X. Ma, X. Gao, and J. Tang, “Distributed Energy-Efficient Multi-UAV Navigation for Long-Term Communication Coverage by Deep Reinforcement Learning,” IEEE Transactions on Mobile Computing, vol. 19, no. 6, pp. 1274–1285, 2020. DOI: https://doi.org/10.1109/TMC.2019.2908171
K. Li, W. Ni, E. Tovar, and M. Guizani, “Deep Reinforcement Learning for Real-Time Trajectory Planning in UAV Networks,” International Wireless Communications and Mobile Computing, pp. 958–963, 2020. DOI: https://doi.org/10.1109/IWCMC48107.2020.9148316
K. Li, W. Ni, E. Tovar, and A. Jamalipour, “Deep Q-Learning based Resource Management in UAV-assisted Wireless Powered IoT Networks,” IEEE International Conference on Communications, pp. 1–6, 2020. DOI: https://doi.org/10.1109/ICC40277.2020.9149282
W. Guo, “Partially Explainable Big Data Driven Deep Reinforcement Learning for Green 5G UAV,” IEEE International Conference on Communications pp. 1–7, 2020. DOI: https://doi.org/10.1109/ICC40277.2020.9149151
D. Chen, Y. Wei, L. Wang, C. S. Hong, L. Wang, and Z. Han, “Deep reinforcement learning based strategy for quadrotor UAV pursuer and evader problem,” IEEE International Conference on Communications Workshops, pp. 1–6, 2020. DOI: https://doi.org/10.1109/ICCWorkshops49005.2020.9145456
Y. Zhang, Z. Zhuang, F. Gao, J. Wang, and Z. Han, “Multi-Agent Deep Reinforcement Learning for Secure UAV Communications,” IEEE Wireless Communications and Networking Conference, pp. 1–5, 2020. DOI: https://doi.org/10.1109/WCNC45663.2020.9120592
E. C. Tetila et al., “Automatic Recognition of Soybean Leaf Diseases Using UAV Images and Deep Convolutional Neural Networks,” IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 5, pp. 903–907, 2020. DOI: https://doi.org/10.1109/LGRS.2019.2932385
Q. Liu, L. Shi, L. Sun, J. Li, M. Ding, and F. Shu, “Path Planning for UAV-Mounted Mobile Edge Computing With Deep Reinforcement Learning,” IEEE Transactions on Vehicular Technology, vol. 69, no. 5, pp. 5723–5728, 2020. DOI: https://doi.org/10.1109/TVT.2020.2982508
X. Li, Q. Wang, J. Liu, and W. Zhang, “Trajectory Design and Generalization for UAV Enabled Networks:A Deep Reinforcement Learning Approach,” IEEE Wireless Communications and Networking Conference, pp. 1–6, 2020. DOI: https://doi.org/10.1109/WCNC45663.2020.9120668
F. Ghavimi and R. Jantti, “Energy-Efficient UAV Communications with Interference Management: Deep Learning Framework,” IEEE Wireless Communications and Networking Conference Workshops, pp. 1–6, 2020. DOI: https://doi.org/10.1109/WCNCW48565.2020.9124759
S. Zhou, B. Li, C. Ding, L. Lu, and C. Ding, “An efficient deep reinforcement learning framework for UAVs,” International Symposium on Quality Electronic Design, pp. 323–328, 2020. DOI: https://doi.org/10.1109/ISQED48828.2020.9136980
O. Walker, F. Vanegas, F. Gonzalez, and S. Koenig, “Multi-UAV target-finding in simulated indoor environments using deep reinforcement learning,” IEEE Aerospace Conference, pp. 1–9, 2020. DOI: https://doi.org/10.1109/AERO47225.2020.9172262
R. K. A. Sakir, M. R. Ramli, J. Lee, and D. Kim, “UAV-assisted Real-time Data Processing using Deep Q-Network for Industrial IoTs,” International Conference on Artificial Intelligence in Information and Communication, Feb. 2020, pp. 208–211, 2020.
H. Huang, Y. Yang, H. Wang, Z. Ding, H. Sari, and F. Adachi, “Deep Reinforcement Learning for UAV Navigation Through Massive MIMO Technique,” IEEE Transactions on Vehicular Technology, vol. 69, no. 1, pp. 1117–1121, 2020. DOI: https://doi.org/10.1109/TVT.2019.2952549
N. Gao, Z. Qin, X. Jing, Q. Ni, and S. Jin, “Anti-intelligent UAV jamming strategy via deep q-networks,” IEEE Transactions on Communications, vol. 68, no. 1, pp. 569–581, Jan. 2020. DOI: https://doi.org/10.1109/TCOMM.2019.2947918
N. Zhao, Z. Liu, and Y. Cheng, “Multi-Agent Deep Reinforcement Learning for Trajectory Design and Power Allocation in Multi-UAV Networks,” IEEE Access, vol. 8, pp. 139670–139679, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.3012756
J. Zhang, Z. Yu, S. Mao, S. C. G. Periaswamy, J. Patton, and X. Xia, “IADRL: Imitation Augmented Deep Reinforcement Learning Enabled UGV-UAV Coalition for Tasking in Complex Environments,” IEEE Access, vol. 8, pp. 102335–102347, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2997304
Q. Yang, J. Zhang, G. Shi, J. Hu, and Y. Wu, “Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning,” IEEE Access, vol. 8, pp. 363–378, 2020. DOI: https://doi.org/10.1109/ACCESS.2019.2961426
Q. Wu, Y. Zhou, X. Wu, G. Liang, Y. Ou, and T. Sun, “Real-time running detection system for UAV imagery based on optical flow and deep convolutional networks,” IET Intelligent Transport Systems, vol. 14, no. 5, pp. 278–287, 2020. DOI: https://doi.org/10.1049/iet-its.2019.0455
J. Tang, J. Song, J. Ou, J. Luo, X. Zhang, and K. Wong, “Minimum Throughput Maximization for Multi-UAV Enabled WPCN: A Deep Reinforcement Learning Method,” IEEE Access, vol. 8, pp. 9124–9132, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2964042
H. Qi, Z. Hu, H. Huang, X. Wen, and Z. Lu, “Energy Efficient 3-D UAV Control for Persistent Communication Service and Fairness: A Deep Reinforcement Learning Approach,” IEEE Access, vol. 8, pp. 53172–53184, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2981403
B. Li and Y. Wu, “Path Planning for UAV Ground Target Tracking via Deep Reinforcement Learning,” IEEE Access, vol. 8, pp. 29064–29074, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2971780
X. Hu and D. Li, “Research on a Single-Tree Point Cloud Segmentation Method Based on UAV Tilt Photography and Deep Learning Algorithm,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 4111–4120, 2020. DOI: https://doi.org/10.1109/JSTARS.2020.3008918
M. D. Bah, A. Hafiane, and R. Canals, “CRowNet: Deep Network for Crop Row Detection in UAV Images,” IEEE Access, vol. 8, pp. 5189–5200, 2020. DOI: https://doi.org/10.1109/ACCESS.2019.2960873
Q. Wang, W. Zhang, Y. Liu, and Y. Liu, “Multi-UAV Dynamic Wireless Networking With Deep Reinforcement Learning,” IEEE Communications Letters, vol. 23, no. 12, pp. 2243–2246, . 2019. DOI: https://doi.org/10.1109/LCOMM.2019.2940191
K. Li, W. Ni, E. Tovar, and A. Jamalipour, “On-Board Deep Q-Network for UAV-Assisted Online Power Transfer and Data Collection,” IEEE Transactions on Vehicular Technology, vol. 68, no. 12, pp. 12215–12226, 2019. DOI: https://doi.org/10.1109/TVT.2019.2945037
B. Kellenberger, D. Marcos, S. Lobry, and D. Tuia, “Half a Percent of Labels is Enough: Efficient Animal Detection in UAV Imagery Using Deep CNNs and Active Learning,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 12, pp. 9524–9533, 2019. DOI: https://doi.org/10.1109/TGRS.2019.2927393
W. Zhang, K. Song, X. Rong, and Y. Li, “Coarse-to-Fine UAV Target Tracking With Deep Reinforcement Learning,” IEEE Transactions on Automation Science and Engineering, vol. 16, no. 4, pp. 1522–1530, 2019. DOI: https://doi.org/10.1109/TASE.2018.2877499
A. M. Koushik, F. Hu, and S. Kumar, “Deep $Q$ -Learning-Based Node Positioning for Throughput-Optimal Communications in Dynamic UAV Swarm Network,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 3, pp. 554–566, Sep. 2019. DOI: https://doi.org/10.1109/TCCN.2019.2907520
M. Chen, W. Saad, and C. Yin, “Echo-Liquid State Deep Learning for 360° Content Transmission and Caching in Wireless VR Networks With Cellular-Connected UAVs,” IEEE Transactions on Communications, vol. 67, no. 9, pp. 6386–6400, Sep. 2019. DOI: https://doi.org/10.1109/TCOMM.2019.2917440
U. Challita, W. Saad, and C. Bettstetter, “Interference Management for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach,” IEEE Transactions on Wireless Communications, vol. 18, no. 4, pp. 2125–2140, Apr. 2019. DOI: https://doi.org/10.1109/TWC.2019.2900035
C. Wang, J. Wang, Y. Shen, and X. Zhang, “Autonomous Navigation of UAVs in Large-Scale Complex Environments: A Deep Reinforcement Learning Approach,” IEEE Transactions on Vehicular Technology, vol. 68, no. 3, pp. 2124–2136, Mar. 2019. DOI: https://doi.org/10.1109/TVT.2018.2890773
V. N. Nguyen, R. Jenssen, and D. Roverso, “Intelligent Monitoring and Inspection of Power Line Components Powered by UAVs and Deep Learning,” IEEE Power and Energy Technology Systems Journal, vol. 6, no. 1, pp. 11–21, Mar. 2019. DOI: https://doi.org/10.1109/JPETS.2018.2881429
L. Zhu, S. Zhang, H. Zhao, S. Chen, D. Wei, and X. Lu, “Classification of UAV-to-Ground Vehicles Based on Micro-Doppler Signatures Using Singular Value Decomposition and Deep Convolutional Neural Networks,” IEEE Access, vol. 7, pp. 22133–22143, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2898642
C. Wu et al., “UAV Autonomous Target Search Based on Deep Reinforcement Learning in Complex Disaster Scene,” IEEE Access, vol. 7, pp. 117227–117245, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2933002
R. Geraldes et al., “UAV-Based Situational Awareness System Using Deep Learning,” IEEE Access, vol. 7, pp. 122583–122594, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2938249
J. Zhu et al., “Urban Traffic Density Estimation Based on Ultrahigh-Resolution UAV Video and Deep Neural Network,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 12, pp. 4968–4981, 2018. DOI: https://doi.org/10.1109/JSTARS.2018.2879368
W. Shafik, S. M. Matinkhah, M. N. Sanda and F. Shokoor, “Internet of Things-Based Energy Efficiency Optimization Model in Fog Smart Cities,” JOIV: International Journal on Informatics Visualization, vol. 5, no. 2, pp. 105-112, 2021. DOI: https://doi.org/10.30630/joiv.5.2.373
Z. Fan, J. Lu, M. Gong, H. Xie, and E. D. Goodman, “Automatic Tobacco Plant Detection in UAV Images via Deep Neural Networks,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 3, pp. 876–887, Mar. 2018. DOI: https://doi.org/10.1109/JSTARS.2018.2793849
A. Zeggada, F. Melgani, and Y. Bazi, “A Deep Learning Approach to UAV Image Multilabeling,” IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 5, pp. 694–698, May 2017. DOI: https://doi.org/10.1109/LGRS.2017.2671922
A. Predescu, D. Arsene, B. Pahonțu, M. Mocanu and C. Chiru, “A Serious Gaming Approach for Crowdsensing in Urban Water Infrastructure with Blockchain Support,” Applied Sciences, vol. 11, no. 4, pp. 1449, 2021. DOI: https://doi.org/10.3390/app11041449
W. Shafik, S. M. Matinkhah, and M. Ghasemzadeh, “A Fast Machine Learning for 5G Beam Selection for Unmanned Aerial Vehicle Applications,” Inf. Syst. Telecommun., vol. 262, 2019.
Y. Chen, H. Tang, J. Wang and J. Song, “Optimizing Age Penalty in Time-Varying Networks with Markovian and Error-Prone Channel State,” Entropy, vol. 23, no. 1, pp. 91, 2021. DOI: https://doi.org/10.3390/e23010091
G. L. Anil and J. M. Iqbal, “Secure energy efficient network priority routing protocol for effective data collection and key management in dynamic WSNs,” International Journal of Information and Computer Security, vol. 14, no. 1, pp. 60-78, 2021. DOI: https://doi.org/10.1504/IJICS.2021.112208
M. Fathy, K. A. Abdel-Moein, W. A. Osman, M. A. Erfan, A. Prince, et al., “Performance of different laboratory methods for detection of clostridium difficile in animal samples,” Adv. Anim. Vet. Sci., vol. 9, no. 1, pp. 132-136, 2021. DOI: https://doi.org/10.17582/journal.aavs/2021/9.1.132.136
Y. Fu, Z. Li, W. Wang, H. He, F. Xiong, et al., “Robust Coarse-to-Fine Registration Scheme for Mobile Laser Scanner Point Clouds Using Multiscale Eigenvalue Statistic-Based Descriptor,” Sensors, vol. 21, no. 7, pp. 2431, 2021. DOI: https://doi.org/10.3390/s21072431
B. Ravi, J. Thangaraj, “Stochastic traffic flow modeling for multi-hop cooperative data dissemination in VANETs,” Physical Communication, vol. 46, pp. 101290, 2021. DOI: https://doi.org/10.1016/j.phycom.2021.101290
C. Xu, D. Li, Q. Chen, M. Liu and K. Meng, “Joint trajectory and transmission optimization for energy efficient UAV enabled eLAA network,” Ad Hoc Networks, vol. 116, pp.102466, 2021. DOI: https://doi.org/10.1016/j.adhoc.2021.102466
K. Wang, W. Heng, X. Li and J. Wu, “Interference Management and Resource Allocation in Multi-Channel Ad Hoc Cognitive Radio Network,” IEICE Transactions on Communications, vol. 104, no. 3, pp. 320-327. DOI: https://doi.org/10.1587/transcom.2020EBP3103
F. Meng, B. Xu, T. Zhang, B. Muthu, C. B. Sivaparthipan, “Application of AI in image recognition technology for power line inspection,” Energy Systems, vol. 4, pp. 1-23, 2021. DOI: https://doi.org/10.1007/s12667-020-00414-8
L. Sieber, C. Ruch, S. Hörl, K. W. Axhausen and E. Frazzoli, "Improved public transportation in rural areas with self-driving cars: a study on the operation of swiss train lines," Transportation research part A: policy and practice, vol. 134, pp. 35-51, 2020. DOI: https://doi.org/10.1016/j.tra.2020.01.020
S. M. Matinkhah, W. Shafik and M. Ghasemzadeh, “Emerging Artificial Intelligence Application: Reinforcement Learning Issues on Current Internet of Things,” 16th international Conference in information knowledge and Technology, Tehran, Iran, 2019.
Y. Jun, A. Craig, W. Shafik and L. Sharif, “Artificial Intelligence Application in Cybersecurity and Cyberdefense,” Wireless Communications and Mobile Computing, 2021. DOI: https://doi.org/10.1155/2021/3329581
R. E. Haas, S. Bhattacharjee and D. P. Möller, "Advanced Driver Assistance Systems," In Smart Technologies, Springer, Singapore, pp. 345-371, 2020. DOI: https://doi.org/10.1007/978-981-13-7139-4_27
R. Muthalagu, A. Bolimera and V. Kalaichelvi, "Lane detection technique based on perspective transformation and histogram analysis for self-driving cars," Computers & Electrical Engineering, vol. 85, pp. 106653, 2020. DOI: https://doi.org/10.1016/j.compeleceng.2020.106653
W. Shafik, S. M. Matinkhah, “How to use Erlang B to determine the blocking probability of packet loss in a wireless communication,” 13th Symposium on Advances in Science & Technology, Mashhad, Iran, 2018.
S. Sripranav, A. Ravi, K. Gautham and R. L. Velusamy, "Smart traffic navigation system (STNS) to reduce travel time by integrating signals from navigation and traffic systems," Smart Trends in Computing and Communications: Proceedings of SmartCom, Springer, Singapore, pp. 101-111, 2020. DOI: https://doi.org/10.1007/978-981-15-5224-3_10
T. T. Krishan, R. S. Alkhawaldeh, I. Al-Hadid, R. Al Azawi and S. H. Al-Sharaeh, "An impact of smart traffic sensing on strategic planning for sustainable smart cities," In Sustainable Development and Social Responsibility, vol. 2, pp. 25-31, 2020. DOI: https://doi.org/10.1007/978-3-030-32902-0_4
W. Shafik, M. Matinkhah, M. Asadi, Z. Ahmadi, Z. Hadiyan, “A study on internet of things performance evaluation,” Journal of Communications Technology, Electronics and Computer Science, pp. 1-9, 2020.
W. Shafik, S. M. Matinkhah, “Privacy issues in social Web of things,” IEEE International Conference on Web Research, pp. 208-214, 2019. DOI: https://doi.org/10.1109/ICWR.2019.8765254
Y. Huang, X. Xiang, H. Zhou, D. Tang and Y. Sun, “Online Identification-Verification-Prediction Method for Parallel System Control of UAVs,” Aerospace, vol. 8, no. 4, pp. 99, 2021. DOI: https://doi.org/10.3390/aerospace8040099
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 EAI Endorsed Transactions on Internet of Things
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
