Systematic Review of Regression Algorithms for Predictive Analytics
Article Sidebar
Main Article Content
Abstract
This systematic review provides an in-depth examination of regression algorithms applied in predictive analytics across multiple domains, including engineering, transportation, and education. The study synthesizes developments in both traditional statistical and modern machine learning (ML) regression models, outlining their evolution, taxonomy, and comparative performance. Traditional multiple linear regression (MLR) has been foundational for predictive analysis but struggles with scalability and computational efficiency in the era of big data. To address these limitations, adaptations such as MapReduce-based MLR have been proposed, enabling distributed computation for large-scale data processing. Simultaneously, hybrid and ML-based regression models—such as support vector regression (SVR), regression trees (CART), and integrated linear–deep learning frameworks—demonstrate superior performance in modeling complex, non-linear relationships.
This review establishes a structured taxonomy differentiating regression approaches based on assumptions, computational properties, and domain-specific requirements, highlighting the balance between model interpretability and predictive power. Comparative evaluations using metrics such as variance explained (R²), mean squared error (MSE), and computational scalability reveal that hybrid regression models often outperform conventional techniques in accuracy and adaptability. The review also identifies persistent challenges related to big data management, model interpretability, data harmonization, and real-time scalability. Emerging research trends emphasize explainable AI (XAI), ensemble regression strategies, and enhanced distributed computing frameworks to overcome these obstacles. Ultimately, this study contributes a holistic understanding of regression-based predictive analytics, guiding researchers and practitioners toward developing scalable, interpretable, and domain-adaptive models for next-generation data-driven decision-making.
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
P. Bilski, “Analysis of the ensemble of regression algorithms for the analog circuit parametric identification,” Measurement (Lond), vol. 160, 2020, doi: 10.1016/j.measurement.2020.107829. DOI: https://doi.org/10.1016/j.measurement.2020.107829
S. Marasco, G. C. Marano, and G. P. Cimellaro, “Evolutionary polynomial regression algorithm combined with robust bayesian regression,” Advances in Engineering Software, vol. 167, 2022, doi: 10.1016/j.advengsoft.2022.103101. DOI: https://doi.org/10.1016/j.advengsoft.2022.103101
A. D. Siburian et al., “Laptop Price Prediction with Machine Learning Using Regression Algorithm,” Jurnal Sistem Informasi dan Ilmu Komputer Prima(JUSIKOM PRIMA), vol. 6, no. 1, 2022, doi: 10.34012/jurnalsisteminformasidanilmukomputer.v6i1.2850. DOI: https://doi.org/10.34012/jurnalsisteminformasidanilmukomputer.v6i1.2850
S. Raizada and J. R. Saini, “Comparative Analysis of Supervised Machine Learning Techniques for Sales Forecasting,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 11, 2021, doi: 10.14569/IJACSA.2021.0121112. DOI: https://doi.org/10.14569/IJACSA.2021.0121112
N. R. Setyoningrum, P. J. Rahimma, S. T. Teknologi, I. Tanjungpinang, and K. Tanjungpinang, “Implementasi Algoritma Regresi Linear Dalam Sistem Prediksi Pendaftar Mahasiswa Baru Sekolah Tinggi Teknologi Indonesia Tanjungpinang,” Prosiding Seminar Nasional Ilmu Sosial dan Teknologi (SNISTEK), no. 4, 2022.
Z. Nie, X. Bai, L. Nie, and J. Wu, “Optimization of the Economic and Trade Management Legal Model Based on the Support Vector Machine Algorithm and Logistic Regression Algorithm,” Math Probl Eng, vol. 2022, 2022, doi: 10.1155/2022/4364295. DOI: https://doi.org/10.1155/2022/4364295
J. Fei, Z. Wu, X. Sun, D. Su, and X. Bao, “Research on tunnel engineering monitoring technology based on BPNN neural network and MARS machine learning regression algorithm,” Neural Comput Appl, vol. 33, no. 1, 2021, doi: 10.1007/s00521-020-04988-3. DOI: https://doi.org/10.1007/s00521-020-04988-3
C. Shi, C. Ji, H. Wang, S. Wang, J. Yang, and Y. Ge, “Comparative evaluation of intelligent regression algorithms for performance and emissions prediction of a hydrogen-enriched Wankel engine,” Fuel, vol. 290, 2021, doi: 10.1016/j.fuel.2020.120005. DOI: https://doi.org/10.1016/j.fuel.2020.120005
K. S. Priya, “Linear Regression Algorithm in Machine Learning through MATLAB,” Int J Res Appl Sci Eng Technol, vol. 9, no. 12, 2021, doi: 10.22214/ijraset.2021.39410. DOI: https://doi.org/10.22214/ijraset.2021.39410
S. M. Almufti, R. B. Marqas, Z. A. Nayef, and T. S. Mohamed, “Real Time Face-mask Detection with Arduino to Prevent COVID-19 Spreading,” Qubahan Academic Journal, vol. 1, no. 2, pp. 39–46, Apr. 2021, doi: 10.48161/qaj.v1n2a47. DOI: https://doi.org/10.48161/qaj.v1n2a47
H. Huang, X. Wei, and Y. Zhou, “An overview on twin support vector regression,” Neurocomputing, vol. 490, 2022, doi: 10.1016/j.neucom.2021.10.125. DOI: https://doi.org/10.1016/j.neucom.2021.10.125
D. Haryadi, A. R. Hakim, D. M. U. Atmaja, and S. N. Yutia, “Implementation of Support Vector Regression for Polkadot Cryptocurrency Price Prediction,” International Journal on Informatics Visualization, vol. 6, no. 1–2, 2022, doi: 10.30630/joiv.6.1-2.945. DOI: https://doi.org/10.30630/joiv.6.1-2.945
Y. Kenzhebek, T. Imankulov, D. Akhmed-Zaki, and B. Daribayev, “IMPLEMENTATION OF REGRESSION ALGORITHMS FOR OIL RECOVERY PREDICTION,” Eastern-European Journal of Enterprise Technologies, vol. 2, no. 2–116, 2022, doi: 10.15587/1729-4061.2022.253886. DOI: https://doi.org/10.15587/1729-4061.2022.253886
R. D. Cook and L. Forzani, “PLS regression algorithms in the presence of nonlinearity,” Chemometrics and Intelligent Laboratory Systems, vol. 213, 2021, doi: 10.1016/j.chemolab.2021.104307. DOI: https://doi.org/10.1016/j.chemolab.2021.104307
I. H. Rahmana, A. R. Febriyani, I. Ranggadara, Suhendra, and I. S. Karima, “Comparative study of extraction features and regression algorithms for predicting drought rates,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 20, no. 3, 2022, doi: 10.12928/TELKOMNIKA.v20i3.23156. DOI: https://doi.org/10.12928/telkomnika.v20i3.23156
T. Wahyudi and D. S. Arroufu, “IMPLEMENTATION OF DATA MINING PREDICTION DELIVERY TIME USING LINEAR REGRESSION ALGORITHM,” Journal of Applied Engineering and Technological Science, vol. 4, no. 1, 2022, doi: 10.37385/jaets.v4i1.918. DOI: https://doi.org/10.37385/jaets.v4i1.918
W. Li, W. Wang, and W. Huo, “RegBoost: a gradient boosted multivariate regression algorithm,” International Journal of Crowd Science, vol. 4, no. 1, 2020, doi: 10.1108/IJCS-10-2019-0029. DOI: https://doi.org/10.1108/IJCS-10-2019-0029
I. Papailiou, F. Spyropoulos, I. Trichakis, and G. P. Karatzas, “Artificial Neural Networks and Multiple Linear Regression for Filling in Missing Daily Rainfall Data,” Water (Switzerland), vol. 14, no. 18, 2022, doi: 10.3390/w14182892. DOI: https://doi.org/10.3390/w14182892
M. Flores-Sosa, E. Avilés-Ochoa, J. M. Merigó, and J. Kacprzyk, “The OWA operator in multiple linear regression,” Appl Soft Comput, vol. 124, 2022, doi: 10.1016/j.asoc.2022.108985. DOI: https://doi.org/10.1016/j.asoc.2022.108985
W. A. L. Sinaga, S. Sumarno, and I. P. Sari, “The Application of Multiple Linear Regression Method for Population Estimation Gunung Malela District,” JOMLAI: Journal of Machine Learning and Artificial Intelligence, vol. 1, no. 1, 2022, doi: 10.55123/jomlai.v1i1.143. DOI: https://doi.org/10.55123/jomlai.v1i1.143
S. M. Almufti, R. R. Asaad, and B. W. Salim, “Review on Elephant Herding Optimization Algorithm Performance in Solving Optimization Problems,” Article in International Journal of Engineering and Technology, vol. 7, no. 4, pp. 6109–6114, 2018, doi: 10.14419/ijet.v7i4.23127. DOI: https://doi.org/10.14419/ijet.v7i4.28473
K. V. Sheelavathy and V. Udaya Rani, “Detection IoT attacks using Lasso regression algorithm with ensemble classifier,” International Journal of Pervasive Computing and Communications, vol. 21, no. 1, 2025, doi: 10.1108/IJPCC-09-2022-0316. DOI: https://doi.org/10.1108/IJPCC-09-2022-0316
J. A. Clarin, “Comparison of the Performance of Several Regression Algorithms in Predicting the Quality of White Wine in WEKA,” International Journal of Emerging Technology and Advanced Engineering, vol. 12, no. 7, 2022, doi: 10.46338/ijetae0722_03. DOI: https://doi.org/10.46338/ijetae0722_03
W. C. Tsai, “A hybrid taguchi-regression algorithm for a fuel injection control system,” Sensors, vol. 22, no. 1, 2022, doi: 10.3390/s22010277. DOI: https://doi.org/10.3390/s22010277
N. H. Dardas and N. D. Georganas, “Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques,” IEEE Trans Instrum Meas, vol. 60, no. 11, pp. 3592–3607, Nov. 2011, doi: 10.1109/TIM.2011.2161140. DOI: https://doi.org/10.1109/TIM.2011.2161140
Z. Cui and G. Gong, “The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features,” Neuroimage, vol. 178, 2018, doi: 10.1016/j.neuroimage.2018.06.001. DOI: https://doi.org/10.1016/j.neuroimage.2018.06.001
M. Chen, C. Yu, G. Guo, and S. Lin, “Faster quantum ridge regression algorithm for prediction,” International Journal of Machine Learning and Cybernetics, vol. 14, no. 1, 2023, doi: 10.1007/s13042-022-01526-6. DOI: https://doi.org/10.1007/s13042-022-01526-6
A. F. Abate, P. Barra, C. Pero, and M. Tucci, “Head pose estimation by regression algorithm,” Pattern Recognit Lett, vol. 140, 2020, doi: 10.1016/j.patrec.2020.10.003. DOI: https://doi.org/10.1016/j.patrec.2020.10.003
Y. D. Almoallem, I. B. M. Taha, M. I. Mosaad, L. Nahma, and A. Abu-Siada, “Application of logistic regression algorithm in the interpretation of dissolved gas analysis for power transformers,” Electronics (Switzerland), vol. 10, no. 10, 2021, doi: 10.3390/electronics10101206. DOI: https://doi.org/10.3390/electronics10101206