A Huber-Regularized Latent Factorization of Tensors Approach for Large-Scale Incomplete Water Quality Data Imputation
DOI:
https://doi.org/10.4108/airo.14233Keywords:
Water Quality Data Imputation, High-Dimensional and Incomplete Tensor, Tucker Decomposition, Latent Factorization of Tensors, Huber RegularizationAbstract
An online water quality monitoring network continuously collects multiple key water quality indicators across distributed monitoring sites via wireless sensors, providing essential data support for intelligent water resource management and ecological assessment. These large-scale observations can be naturally represented as a high-dimensional and incomplete (HDI) water quality tensor with frequent missing entries. Such an HDI tensor contains rich multivariate, spatial, and temporal dependencies that can be effectively exploited for missing data recovery. Latent factorization of tensors (LFT) models have demonstrated remarkable capability in extracting latent knowledge from HDI tensors, thereby enabling accurate missing data imputation. However, existing LFT models generally adopt the conventional L2 regularization scheme, making them highly sensitive to sensor outliers that are ubiquitous in real-time monitoring environments. To address this issue, this study proposes a Huber-regularized latent factorization of tensors (HRL) model under the Tucker decomposition framework. Specifically, a Huber regularization scheme is incorporated into the learning objective to improve robustness against outliers while preserving the smoothness of latent factor learning. Empirical studies on eight practical HDI water quality tensors demonstrate that the proposed HRL model consistently outperforms state-of-the-art models in terms of computational efficiency and imputation accuracy, offering an effective and robust solution for large-scale incomplete water quality data reconstruction and facilitating reliable downstream decision-making.
Downloads
References
[1] X. Wu, L. Wang, M. Ge, J. Jiang, Y. Cai, and B. Yang, “Adaptive biases-incorporated latent factorization of tensors for predicting missing data in water quality monitoring networks,” Front. Phys. 2025;13:1587012.
[2] X. Wu, L. Wang, M. Xie, and K. Shan, “A well-designed regularization scheme for latent factorization of high-dimensional and incomplete water-quality tensors from sensor networks,” in Proc. 19th IEEE Int. Conf. Mobility, Sensing and Networking. 2023. p. 596–603.
[3] R. Marcé, G. George, P. Buscarinu, M. Deidda, J. Dunalska, E. de Eyto, G. Flaim, H.-P. Grossart, V. Istvanovics, M. Lenhardt, E. Moreno-Ostos, B. Obrador, I. Ostrovsky, D. C. Pierson, J. Potužák, S. Poikane, K. Rinke, S. Rodríguez-Mozaz, P. A. Staehr, K. Šumberová, G. Waajen, G. A. Weyhenmeyer, K. C. Weathers, M. Zion, B. W. Ibelings, and E. Jennings, “Automatic high frequency monitoring for improved lake and reservoir management,” Environ. Sci. Technol. 2016;50(20):10780–10794.
[4] J. He, Z. Wang, W. Huang, F. Zhou, A. Wang, X. Wu, P. Chen, T. He, J. Gan, P. Wei, Z. Li, and C. Wu, “An efficient hybrid model with Harris Hawks optimization algorithm for predicting oat water,” EAI Endorsed Trans. AI and Robotics 2026;5.
[5] A. Al Mamun, M. F. S. Tanoor, A. K. M. Masum, T. Bhuiyan, and M. M. Hassan, “Explainable AI for customer churn prediction in the energy sector using ensemble machine learning models,” EAI Endorsed Trans. AI and Robotics 2026;5.
[6] O. Dagtekin and N. Dethlefs, “Imputation of partially observed water quality data using self-attention LSTM,” in Proc. IEEE Int. Joint Conf. Neural Networks (IJCNN). 2022. p. 1–8.
[7] J. Choi, K. J. Lim, and B. J. Ji, “Robust imputation method with context-aware voting ensemble model for management of water-quality data,” Water Res. 2023;243:120369.
[8] Y. Zhang and P. Thorburn, “Handling missing data in near real-time environmental monitoring: a system and a review of selected methods,” Future Gener. Comput. Syst. 2022;128:63–72.
[9] S. Ouyang, “Evaluation of river water quality monitoring stations by principal component analysis,” Water Res. 2005;39(12):2621–2635.
[10] X. Wu, K. Shan, L. Wang, J. Wang, and M. Shang, “Spa-tiotemporal water quality data reconstruction: A tensor factorization framework,” Ecol. Inform. 2025;90:103283.
[11] E. E. Papalexakis, C. Faloutsos, and N. D. Sidiropoulos, “Tensors for data mining and data fusion: Models, applications, and scalable algorithms,” ACM Trans. Intell. Syst. Technol. 2017;8(2):art. 16, pp. 1–44.
[12] X. Luo, M. Zhou, S. Li, Z. You, Y. Xia, and Z. Feng, “A nonnegative latent factor model for large-scale sparse matrices in recommender systems and its application,” IEEE Trans. Neural Netw. Learn. Syst. 2016;27(3):579–592.
[13] Y. Ji, Q. Wang, X. Li, and J. Liu, “A survey on tensor techniques and applications in machine learning,” IEEE Access 2019;7:162950–162990.
[14] Y. Wu, H. Tan, Y. Li, J. Zhang, and X. Chen, “A fused CP factorization method for incomplete tensors,” IEEE Trans. Neural Netw. Learn. Syst. 2019;30(3):751–764.
[15] W. Zhang, H. Sun, X. Liu, and X. Guo, “Temporal QoS-aware Web service recommendation via non-negative tensor factorization,” in Proc. 23rd Int. Conf. World Wide Web. 2014. p. 585–596.
[16] X. Wu, K. Shan, F. Recknagel, L. Wang, and M. Shang, “Enhanced tensor factorization for spatiotemporal imputation of high-frequency water quality monitoring data,” Environ. Model. Softw. 2025;193:Art. no. 106667.
[17] E. Acar, D. Dunlavy, T. Kolda, and M. Morup, “Scalable tensor factorizations with missing data,” in Proc. SIAM Int. Conf. Data Mining. 2010. p. 701–712.
[18] Q. L. Liu, L. Z. Lu, and Z. Chen, “Non-negative Tucker decomposition with graph regularization and smooth constraint for clustering,” Pattern Recognit. 2023;148:110207.
[19] S. Fang, A. Narayan, R. M. Kirby, and S. Zhe, “Bayesian continuous-time Tucker decomposition,” in Proc. 39th Int. Conf. Mach. Learn. (ICML). PMLR; 2022. p. 6235–6245.
[20] Y. Zhang, Y. Liu, and Y. Wang, “Displacement data imputation in urban Internet of Things system based on Tucker decomposition with L2 regularization,” IEEE Internet Things J. 2022;9(18):13315–13326.
[21] P. Tang, T. Ruan, H. Wu, and X. Luo, “Temporal pattern-aware QoS prediction by Biased Non-negative Tucker Fac-torization of tensors,” Neurocomputing 2024;582:127447.
[22] P. J. Huber, “Robust estimation of a location parameter,” Ann. Math. Stat. 1964;35(1):73–101.
[23] P. Ochs, A. Dosovitskiy, T. Brox, and T. Pock, “On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision,” SIAM J. Imaging Sci. 2015;8(1):331–372.
[24] L. R. Tucker, “Implications of factor analysis of three-way matrices for measurement of change,” in Problems in Measuring Change, C. W. Harris, editor. Madison, WI, USA: Univ. Wis. Press; 1963. p. 122–137.
[25] L. R. Tucker, “Some mathematical notes on three-mode factor analysis,” Psychometrika 1966;31(3):279–311.
[26] N. D. Sidiropoulos, L. De Lathauwer, X. Fu, K. Huang, E. E. Papalexakis, and C. Faloutsos, “Tensor decomposition for signal processing and machine learning,” IEEE Trans. Signal Process. 2017;65(13):3551–3582.
[27] P. J. Huber and E. M. Ronchetti, Robust Statistics. 2nd ed. Hoboken, NJ, USA: Wiley; 2009.
[28] Y. Koren and R. Bell, “Advances in collaborative filtering,” in Recommender Systems Handbook, F. Ricci, L. Rokach, and B. Shapira, editors. Boston, MA, USA: Springer; 2015. p. 77–118.
[29] A. Paterek, “Improving regularized singular value decomposition for collaborative filtering,” in Proc. KDD Cup Workshop. 2007. p. 39–42.
[30] R. Gemulla, E. Nijkamp, P. J. Haas, and Y. Sismanis, “Large-scale matrix factorization with distributed stochas-tic gradient descent,” in Proc. 17th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min.. 2011. p. 69–77.
[31] S. Wang, Y. Ma, B. Cheng, F. Yang, and R. Chang, “Multi-dimensional QoS prediction for service recommenda-tions,” IEEE Trans. Serv. Comput. 2019;12(1):47–57.
[32] X. Su, M. Zhang, Y. Liang, Z. Cai, L. Guo, and Z. Ding, “A tensor-based approach for the QoS evaluation in service-oriented environments,” IEEE Trans. Netw. Serv. Manag. 2021;18(3):3843–3857.
[33] H. Che, B. Pan, M.-F. Leung, Y. Cao, and Z. Yan, “Tensor factorization with sparse and graph regularization for fake news detection on social networks,” IEEE Trans. Comput. Soc. Syst. 2024;11(4):4888–4898.
[34] H. Wu, Y. Qiao, and X. Luo, “A fine-grained regularization scheme for nonnegative latent factorization of high-dimensional and incomplete tensors,” IEEE Trans. Serv. Comput. 2024;17(6):3006–3021.
[35] V .-K. Pham, M.-D. Do, T.-N. Dang, and L. Tran, “Hardware-aware INT8 quantization and FPGA deploy-ment of MobileNetV2 for real-time facial landmark detection,” EAI Endorsed Trans. AI and Robotics 2026.
[36] S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding,” in Proc. 4th Int. Conf. Learn. Represent. (ICLR). 2016.
[37] K. K. M. Mickelin and S. Wahls, “Tensor Ring Decomposition for Efficient Subspace Projections,” SIAM J. Matrix Anal. Appl. 2022;43(2):720–743.
[38] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. 3rd Int. Conf. Learn. Represent. (ICLR). 2015.
[39] J. Duchi, E. Hazan, and Y. Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” J. Mach. Learn. Res. 2011;12:2121–2159.
[40] K. Kalaivani, P. Deepan, G. Ganesh, J. Ravichandran, and S. Dhiravidaselvi, “Hyperband-optimized convolutional neural network model for efficient brain tumor classification and prediction,” EAI Endorsed Trans. AI and Robotics 2025.
[41] J. Snoek, H. Larochelle, and R. P. Adams, “Practical Bayesian optimization of machine learning hyperparam-eters,” in Proc. 25th Int. Conf. Neural Inf. Process. Syst. (NeurIPS). 2012. p. 2951–2959.
[42] L. Wang, X. Wu, Y. Yu, H. Liu, K. Shan, and B. Yang, “Nonnegative latent factorization of tensors with swish regularization for water quality data imputation,” in Proc. 2024 12th Int. Conf. Commun. Broadband Netw. (ICCBN). 2024. p. 201–207.
[43] J. Demšar, “Statistical comparisons of classifiers over multiple data sets,” J. Mach. Learn. Res. 2006;7:1–30.
[44] X. Wu, J. Hu, Y. Wang, and H. Tu, “AHFM water quality datasets collected from thirteen typical watersheds across China [Data set],” figshare; 2026. [Online]. Available: https://doi.org/10.6084/m9.figshare.32527353
[45] X. Xu, M. Lin, X. Luo, and Z. Xu, “An adaptive bias-extended non-negative latent factorization of tensors model for accurately representing the dynamic QoS data,” IEEE Trans. Serv. Comput. 2025;18(2):603–617.
[46] A. Elragig, A. J. Moshayedi, D. Bassir, Z. Khan, A. Kolahdooz, and A. S. Khan, “Smart agro-ecosystem: A review of LLM-based robotic systems for sensing and decision support in precision agriculture,” EAI Endorsed Trans. AI and Robotics 2026;5.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Hengrui Tu, Xuke Wu, Tinghui Chen, Zhibin Li, Jinli Li
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
This is an open access article distributed under the terms of the CC BY-NC-SA 4.0, which permits copying, redistributing, remixing, transformation, and building upon the material in any medium so long as the original work is properly cited.
