Water Quality Estimation and Anomaly Detection: A Review
Keywords:Water quality, Anomaly detection, Water management systems, Water analytics, Water distribution networks, Machine learning
Critical infrastructures that provide irreplaceable services are systems that contain industrial control systems (ICS) that can cause great economic losses, security vulnerabilities and disruption of public order when the information in it is corrupted. These ICSs, which were previously isolated, have now become systems that contain online sensors, wireless networks and artificial intelligence technologies. This situation has also increased the scope of attacks by malicious people who intend to carry out industrial espionage and sabotage these systems. In this study, water quality estimation systems and anomaly detection are comprehensively examined. In this direction, the statistics of the studies in the literature, the methods for water quality anomaly detection, the existing data sets, and the difficulties encountered in the water systems to achieve better water management are discussed. Principle findings of this research can be summarized as follows: (i) new methodologies and architectures have improved water quality assessment through anomaly detection, (ii) different datasets including multi-modal information have been presented, and (iii) remaining challenges and prospects have been investigated.
Türkiye’nin su politikaları, t.c. dışişleri bakanlığı., https://www.mfa.gov.tr/turkiye_ nin-su-politikasi.tr.mfa. Accessed: 2023-01-12.
Kucukcelebi, C. (2014) Avrupa Birliği Uyum Sürecinde Türkiye’nin Su Politikası, Su Hukuku ve Su Kaynakları Yönetiminde Yeniden Yapılanmalar. Master’s thesis, İstanbul Teknik Üniversitesi.
Health, E. (2013) NSW Guidelines for Drinking Water Management Systems (NSW Government).
Pelekanos, N., Nikolopoulos, D. and Makropoulos, C. (2021) Simulation and vulnerability assessment of water distribution networks under deliberate contamination attacks. Urban Water Journal 18: 1–14. doi:10.1080/1573062X.2020.1864832.
Lu, H. and Ma, X. (2020) Hybrid decision tree-based machine learning models for shortterm water quality prediction. Chemosphere 249. doi:10.1016/j.chemosphere.2020.126169.
Chhipi-Shrestha, G., Mian, H.R., Mohammadiun, S., Rodriguez, M., Hewage, K. and Sadiq, R. (2023) Digital water: artificial intelligence and soft computing applications for drinking water quality assessment. Clean Technologies and Environmental Policy : 1–30.
Berglund, E.Z., Shafiee, M.E., Xing, L. and Wen, J. (2023) Digital twins for water distribution systems. Journal of Water Resources Planning and Management 149(3): 02523001.
Sikder, M.N.K., Nguyen, M.B., Elliott, E.D. and Batarseh, F.A. (2023) Deep h2o: Cyber attacks detection in water distribution systems using deep learning. Journal of Water Process Engineering 52: 103568.
Ismagilova, E., Hughes, L., Dwivedi, Y.K. and Raman, K.R. (2019) Smart cities: Advances in research—an information systems perspective. International Journal of Information Management 47: 88–100.
Bellini, P., Nesi, P. and Pantaleo, G. (2022) Iot-enabled smart cities: A review of concepts, frameworks and key technologies. Applied Sciences 12(3): 1607.
Erdem, T. and Eken, S. (2021) Layer-wise relevance propagation for smart-grid stability prediction. In Mediterranean Conference on Pattern Recognition and Artificial Intelligence (Springer): 315–328.
Breviglieri, P., Erdem, T. and Eken, S. (2021) Predicting smart grid stability with optimized deep models. SN Computer Science 2: 1–12.
Oberascher, M., Rauch, W. and Sitzenfrei, R. (2022) Towards a smart water city: A comprehensive review of applications, data requirements, and communication technologies for integrated management. Sustainable Cities and Society 76: 103442.
Keriwala, N. and Patel, A. (2022) Innovative roadmap for smart water cities: A global perspective. Materials Proceedings 10(1): 1.
Yaacoub, J.P.A., Salman, O., Noura, H.N., Kaaniche, N., Chehab, A. and Malli, M. (2020) Cyber-physical systems security: Limitations, issues and future trends. Microprocessors and microsystems 77: 103201.
Mishra, A. and Ray, A.K. (2022) A novel layered architecture and modular design framework for next-gen cyber physical system. In 2022 International Conference on Computer Communication and Informatics (ICCCI) (IEEE): 1–8.
Latif, S.A., Wen, F.B.X., Iwendi, C., Li-li, F.W., Mohsin, S.M., Han, Z. and Band, S.S. (2022) Ai-empowered, blockchain and sdn integrated security architecture for iot network of cyber physical systems. Computer Communications 181: 274–283.
Manogaran, G., Khalifa, N.E.M., Loey, M. and Taha, M.H.N. (2023) Cyber-Physical Systems for Industrial Transformation: Fundamentals, Standards, and Protocols (CRC Press).
Leigh, C., Alsibai, O., Hyndman, R.J., Kandanaarachchi, S., King, O.C., McGree, J.M., Neelamraju, C. et al. (2019) A framework for automated anomaly detection in high frequency water-quality data from in situ sensors. Science of the Total Environment 664: 885–898.
Li, Z., Liu, H., Zhang, C. and Fu, G. (2023) Generative adversarial networks for detecting contamination events in water distribution systems using multi-parameter, multi-site water quality monitoring. Environmental Science and Ecotechnology 14: 100231.
Fanaee-T, H. and Gama, J. (2016) Tensor-based anomaly detection: An interdisciplinary survey. Knowledge-Based Systems 98: 130–147.
Sebestyen, G. and Hangan, A. (2017) Anomaly detection techniques in cyber-physical systems. Acta Universitatis Sapientiae, Informatica 9(2): 101–118.
Dogo, E.M., Nwulu, N.I., Twala, B. and Aigbavboa, C. (2019) A survey of machine learning methods applied to anomaly detection on drinking-water quality data. Urban Water Journal 16(3): 235–248.
Sagan, V., Peterson, K.T., Maimaitijiang, M., Sidike, P., Sloan, J., Greeling, B.A., Maalouf, S. et al. (2020) Monitoring inland water quality using remote sensing: Potential and limitations of spectral indices, bio-optical simulations, machine learning, and cloud computing. Earth-Science Reviews 205: 103187.
Jiang, J., Zheng, Y., Pang, T., Wang, B., Chachan, R. and Tian, Y. (2020) A comprehensive study on spectral analysis and anomaly detection of river water quality dynamics with high time resolution measurements. Journal of Hydrology 589: 125175.
Ahmed, U., Mumtaz, R., Anwar, H., Mumtaz, S. and Qamar, A.M. (2020) Water quality monitoring: from conventional to emerging technologies. Water Supply 20(1): 28–45.
Gupta, S., Aga, D., Pruden, A., Zhang, L. and Vikesland, P. (2021) Data analytics for environmental science and engineering research. Environmental Science & Technology 55(16): 10895–10907.
Shi, Z., Chow, C.W., Fabris, R., Liu, J. and Jin, B. (2022) Applications of online uv-vis spectrophotometer for drinking water quality monitoring and process control: a review. Sensors 22(8): 2987.
Water quality criteria, https://https://www.epa.gov/wqc. Accessed: 2023-01-12.
Ulusal su planı, t.c. tarım ve orman bakanlığı, https://www.tarimorman.gov.tr/SYGM/Belgeler/NHYP%20DEN%C4%B0Z/ULUSAL%20SU%20PLANI.pdf. Accessed: 2023-01-12.
WHO, W.H.O. (2022) Guidelines for drinking-water quality (World Health Organization WHO).
Kang, G., Gao, J.Z. and Xie, G. (2017) Data-driven water quality analysis and prediction: A survey. 2017 IEEE Third International Conference on Big Data Computing Service and Applications (BigDataService) : 224–232.
Chawla, P., Cao, X., Fu, Y., Hu, C.m.,Wang, M.,Wang, S. and Gao, J. (2021) Water quality prediction of salton sea using machine learning and big data techniques. International Journal of Environmental Analytical Chemistry : 1–24doi:10.1080/03067319.2021.1963713.
Selim, G.E., Hemdan, E.E.D., Shehata, A. and El- Fishawy, N. (2021) Anomaly events classification and detection system in critical industrial internet of things infrastructure using machine learning algorithms. Multimedia Tools and Applications 80: 1–22. doi:10.1007/s11042-020-10354-1.
Nemade, B. and Shah, D. (2022) An efficient iot based prediction system for classification of water using novel adaptive incremental learning framework. J. King Saud Univ. Comput. Inf. Sci. 34(8 Part A): 5121– 5131. doi:10.1016/j.jksuci.2022.01.009, URL https://doi.org/10.1016/j.jksuci.2022.01.009.
Jin, T., Cai, S., Jiang, D. and Liu, J. (2019) A datadriven model for real-time water quality prediction and early warning by an integration method. Environmental Science and Pollution Research 26. doi:10.1007/s11356-019-06049-2.
Page, M.J., McKenzie, J.E., Bossuyt, P.M., Boutron, I., Hoffmann, T.C., Mulrow, C.D., Shamseer, L. et al. (2021) The prisma 2020 statement: an updated guideline for reporting systematic reviews. Systematic reviews 10(1): 1–11.
Van Eck, N.J. and Waltman, L. (2011) Text mining and visualization using vosviewer. arXiv preprint arXiv:1109.2058 .
Mathur, A.P. and Tippenhauer, N.O. (2016) Swat: a water treatment testbed for research and training on ics security. In 2016 International Workshop on Cyberphysical Systems for Smart Water Networks (CySWater): 31–36. doi:10.1109/CySWater.2016.7469060.
Özçelİk, , İskefiyeli, M., Balta, M., Akpinar, K.O. and Toker, F.S. (2021) Center water: A secure testbed infrastructure proposal for waste and potable water management. In 2021 9th International Symposium on Digital Forensics and Security (ISDFS): 1–7. doi:10.1109/ISDFS52919.2021.9486364.
Morris, T., Srivastava, A., Reaves, B., Gao, W., Pavurapu, K. and Reddi, R. (2011) A control system testbed to validate critical infrastructure protection concepts. International Journal of Critical Infrastructure Protection 4: 88–103. doi:10.1016/j.ijcip.2011.06.005.
Gao, H., Peng, Y., Dai, Z., Wang, T., Han, X. and Li, H. (2014) An Industrial Control System Testbed Based on Emulation, Physical Devices and Simulation. In Butts, J. and Shenoi, S. [eds.] 8th International Conference on Critical Infrastructure Protection (ICCIP) (Arlington, United States: Springer), Critical Infrastructure Protection VIII AICT-441: 79–91. doi:10.1007/978-3-662-45355- 1_6, URL https://hal.inria.fr/hal-01386756. Part 1: Control Systems Security.
Ahmed, I., Roussev, V., Johnson, W., Senthivel, S. and Sudhakaran, S. (2016) A scada system testbed for cybersecurity and forensic research and pedagogy: 1–9. doi:10.1145/3018981.3018984.
Taormina, R., Galelli, S., Tippenhauer, N.O., Ostfeld, A. and Salomons, E. (2016) Assessing the effect of cyberphysical attacks on water distribution systems: 436–442. doi:10.1061/9780784479865.046.
Etchevés Miciolino, E., Setola, R., Bernieri, G., Panzieri, S., Pascucci, F. and Polycarpou, M.M. (2017) Fault diagnosis and network anomaly detection in water infrastructures. IEEE Design Test 34(4): 44–51. doi:10.1109/MDAT.2017.2682223.
Kartakis, S., Abraham, E. and McCann, J.A. (2015) Waterbox: A testbed for monitoring and controlling smart water networks. In Proceedings of the 1st ACM International Workshop on Cyber-Physical Systems for Smart Water Networks, CySWater’15 (New York,
NY, USA: Association for Computing Machinery). doi:10.1145/2738935.2738939, URL https://doi.org/10.1145/2738935.2738939.
Itrust labs datasets, https://itrust.sutd.edu.sg/itrust-labs_datasets/dataset_info/. Accessed: 2023-01-12.
Botchkarev, A. (2019) A new typology design of performance metrics to measure errors in machine learning regression algorithms. Interdisciplinary Journal of Information, Knowledge, and Management 14: 045–076. doi:10.28945/4184, URL https://doi.org/10.28945%2F4184.
Parmar, J., Chouhan, S., Raychoudhury, V. and Rathore, S. (2023) Open-world machine learning: applications, challenges, and opportunities. ACM Computing Surveys 55(10): 1–37.
Muharemi, F., Logofătu, D. and Leon, F. (2019) Machine learning approaches for anomaly detection of water quality on a real-world data set. Journal of Information and Telecommunication 3(3): 294–307.
Akpinar, K.O. and Ozcelik, I. (2019) Analysis of machine learning methods in ethercat-based anomaly detection. IEEE Access 7: 184365–184374.
Berry, M.W., Mohamed, A. and Yap, B.W. (2019) Supervised and unsupervised learning for data science (Springer).
Dogo, E.M. (2021) Application of Artificial Intelligence Technologies for Water Quality Anomaly Detection. Ph.D. thesis, University of Johannesburg (South Africa).
Qian, K., Jiang, J., Ding, Y. and Yang, S. (2020) Deep learning based anomaly detection in water distribution systems. In 2020 IEEE International Conference on Networking, Sensing and Control (ICNSC) (IEEE): 1–6.
Luo, Y., Xiao, Y., Cheng, L., Peng, G. and Yao, D. (2021) Deep learning-based anomaly detection in cyberphysical systems: Progress and opportunities. ACM Computing Surveys (CSUR) 54(5): 1–36.
Alom, M.Z., Taha, T.M., Yakopcic, C., Westberg, S., Sidike, P., Nasrin, M.S., Van Esesn, B.C. et al. (2018) The history began from alexnet: A comprehensive survey on deep learning approaches. arXiv preprint arXiv:1803.01164 .
Arias-Rodriguez, L.F., Duan, Z., Díaz-Torres, J.d.J., Basilio Hazas, M., Huang, J., Kumar, B.U., Tuo, Y. et al. (2021) Integration of remote sensing and mexican water quality monitoring system using an extreme learning machine. Sensors 21(12): 4118.
Choi, Y.H., Sadollah, A. and Kim, J.H. (2020) Improvement of cyber-attack detection accuracy from urban water systems using extreme learning machine. Applied Sciences 10(22): 8179.
Li, Y. (2017) Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274 .
Khampuengson, T. and Wang, W. (2022) Deep reinforcement learning ensemble for detecting anomaly in telemetry water level data. Water 14(16): 2492.
Housh, M. and Ohar, Z. (2018) Model-based approach for cyber-physical attack detection in water distribution systems. Water Research 139. doi:10.1016/j.watres.2018.03.039.
Teixeira, M., Salman, T., Zolanvari, M. and Jain, R. (2018) Scada system testbed for cybersecurity research using machine learning approach. Future Internet 10. doi:10.3390/fi10080076.
Phillips, B., Gamess, E. and Krishnaprasad, S. (2020) An evaluation of machine learning-based anomaly detection in a scada system using the modbus protocol. doi:10.1145/3374135.3385282.
Abokifa, A., Haddad, K., Lo, C. and Biswas, P. (2018) Real-time identification of cyber-physical attacks on water distribution systems via machine learning based anomaly detection techniques. Journal of Water Resources Planning and Management 145. doi:10.1061/(ASCE)WR.1943-5452.0001023.
Peterson, K.T., Sagan, V. and Sloan, J.J. (2020) Deep learning-based water quality estimation and anomaly detection using landsat-8/sentinel-2 virtual constellation and cloud computing. GIScience & Remote Sensing 57(4): 510–525. doi:10.1080/15481603.2020.1738061, URL https://doi.org/10.1080/15481603.2020.1738061
Liu, J., Wang, P., Jiang, D., Nan, J. and Zhu, W. (2020) An integrated data-driven framework for surface water quality anomaly detection and early warning. Journal of Cleaner Production 251: 119145. doi:https://doi.org/10.1016/j.jclepro.2019.119145, URL https://www.sciencedirect.com/science/article/ pii/S0959652619340156.
Inoue, J., Yamagata, Y., Chen, Y., Poskitt, C. and Sun, J. (2017) Anomaly detection for a water treatment system using unsupervised machine learning .
Fang, S., Sun, W. and Huang, L. (2019) Anomaly detection for water supply data using machine learning technique. Journal of Physics: Conference Series 1345(2): 022054. doi:10.1088/1742-6596/1345/2/022054, URL https://dx.doi.org/10.1088/1742-6596/1345/2/
Shi, B., Wang, P., Jiang, J. and Liu, R. (2017) Applying high-frequency surrogate measurements and a wavelet-ann model to provide early warnings of rapid surface water quality anomalies. The Science of the total environment 610-611: 1390–1399. doi:10.1016/j.scitotenv.2017.08.232.
Muharemi, F., Logofătu, D. and Leon, F. (2019) Machine learning approaches for anomaly detection of water quality on a real-world data set. Journal of Information and Telecommunication 3(3): 294–307. doi:10.1080/24751839.2019.1565653.
Riss, G., Romano, M., Memon, F. and Kapelan, Z. (2021) Detection of water quality failure events at treatment works using a hybrid two-stage method with cusum and random forest algorithms. Water Supply 21. doi:10.2166/ws.2021.062.
Wu, Z., Chew, A., Meng, X., Cai, J., Pok, J., Kalfarisi, R., Lai, K. et al. (2022) Data-driven and model-based framework for smart water grid anomaly detection and localization. AQUA—Water Infrastructure, Ecosystems and Society 71(1): 31–41.
Thompson, K.A. and Dickenson, E.R. (2021) Using machine learning classification to detect simulated increases of de facto reuse and urban stormwater surges in surface water. Water Research 204: 117556. doi:https://doi.org/10.1016/j.watres.2021.117556, URL
Shah, M.I., Javed, M.F., Alqahtani, A. and Aldrees, A. (2021) Environmental assessment based surface water quality prediction using hyper-parameter optimized machine learning models based on consistent big data. Process Safety and Environmental Protection 151: 324–340. doi:https://doi.org/10.1016/j.psep.2021.05.026, URL https://www.sciencedirect.com/science/article/pii/S0957582021002664.
Macas, M. and Wu, C. (2019) An unsupervised framework for anomaly detection in a water treatment system. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA): 1298–1305. doi:10.1109/ICMLA.2019.00212.
Zhu, J., Liu, X., Shi, Q., He, T., Sun, Z., Guo, X., Liu, W. et al. (2019) Development trends and perspectives of future sensors and mems/nems. Micromachines 11(1): 7.
Mezni, H., Driss, M., Boulila, W., Atitallah, S.B., Sellami, M. and Alharbi, N. (2022) Smartwater: A service-oriented and sensor cloud-based framework for smart monitoring of water environments. Remote Sensing 14(4): 922.
Eken, S., Şara, M., Satılmış, Y., Karslı, M., Tufan, M.F., Menhour, H. and Sayar, A. (2020) A reproducible educational plan to teach mini autonomous race car programming. The International Journal of Electrical Engineering & Education 57(4): 340–360.
Stagge, J.H., Rosenberg, D.E., Abdallah, A.M., Akbar, H., Attallah, N.A. and James, R. (2019) Assessing data availability and research reproducibility in hydrology and water resources. Scientific data 6(1): 1–12.
Li, J., Zhang, C., Zhou, J.T., Fu, H., Xia, S. and Hu, Q. (2021) Deep-lift: deep label-specific feature learning for image annotation. IEEE Transactions on Cybernetics .
Petsiuk, V., Das, A. and Saenko, K. (2018) Rise: Randomized input sampling for explanation of blackbox models. arXiv preprint arXiv:1806.07421 .
Lundberg, S.M. and Lee, S.I. (2017) A unified approach to interpreting model predictions. Advances in neural information processing systems 30.
Das, A. and Rad, P. (2020) Opportunities and challenges in explainable artificial intelligence (xai): A survey. arXiv preprint arXiv:2006.11371 .
Ali, I. (2012) New generation adsorbents for water treatment. Chemical reviews 112(10): 5073–5091.
Dablain, D., Krawczyk, B. and Chawla, N.V. (2022) Deepsmote: Fusing deep learning and smote for imbalanced data. IEEE Transactions on Neural Networks and Learning Systems .
Brentan, B., Carpitella, S., Barros, D., Meirelles, G., Certa, A. and Izquierdo, J. (2021) Water quality sensor placement: a multi-objective and multi-criteria approach. Water Resources Management 35(1): 225–241.
Xue, M., Chew, A.W.Z., Cai, J., Pok, J., Kalfarisi, R. and Wu, Z.Y. (2022) Improving near real-time anomaly event detection and classification with trend change detection for smart water grid operation management. Urban Water Journal : 1–11.
Xu, H., Berres, A., Liu, Y., Allen-Dumas, M.R. and Sanyal, J. (2022) An overview of visualization and visual analytics applications in water resources management. Environmental Modelling & Software : 105396.
Abdallah, A.M., Rheinheimer, D.E., Rosenberg, D.E., Knox, S. and Harou, J.J. (2022) An interoperable software ecosystem to store, visualize, and publish water resources systems modelling data. Environmental Modelling & Software 151: 105371.
Sun, A.Y. and Scanlon, B.R. (2019) How can big data and machine learning benefit environment and water management: a survey of methods, applications, and future directions. Environmental Research Letters 14(7): 073001.
Balta, S., Zavrak, S. and Eken, S. (2022) Real-time monitoring and scalable messaging of scada networks data: A case study on cyber-physical attack detection inwater distribution system. In International Congress of Electrical and Computer Engineering (Springer): 203–215.
Difallah, D.E., Cudre-Mauroux, P. and McKenna, S.A. (2013) Scalable anomaly detection for smart city infrastructure networks. IEEE Internet Computing 17(6): 39–47.
Özgüven, Y.M. and Eken, S. (2021) Distributed messaging and light streaming system for combating pandemics. Journal of Ambient Intelligence and Humanized Computing : 1–15.
Shao, Z., Sumari, N.S., Portnov, A., Ujoh, F., Musakwa, W. and Mandela, P.J. (2021) Urban sprawl and its impact on sustainable urban development: a combination of remote sensing and social media data. Geo-spatial Information Science 24(2): 241–255.
Andreadis, S., Gialampoukidis, I., Bozas, A., Moumtzidou, A., Fiorin, R., Lombardo, F., Karakostas, A. et al. (2021)Watermm: water quality in socialmultimedia task at mediaeval 2021. In Proceedings of the MediaEval 2021 Workshop, Online.
Balta Kaç, S. and Eken, S. (2023) Customer complaintsbased water quality analysis. Water 15(18): 3171.
Ayub, M.A., Ahmad, K., Ahmad, K., Ahmad, N. and Al-Fuqaha, A. (2021) Nlp techniques for water quality analysis in social media content. arXiv preprint arXiv:2112.11441 .
Ahmad, K., Ayub, M., Khan, J., Ahmad, N. and Al-Fuqaha, A. (2022) Social media as an instant source of feedback on water quality. IEEE Transactions on Technology and Society .
Hanif, M., Khawar, A., Tahir, M.A. and Rafi, M. (2021) Deep learning based framework for classification of water quality in social media data. In Proceedings of the MediaEval 2021 Workshop, Online.
How to Cite
Copyright (c) 2023 EAI Endorsed Transactions on Internet of Things
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
Türkiye Bilimsel ve Teknolojik Araştırma Kurumu
Grant numbers 122E610