Systematic Review of Regression Algorithms for Predictive Analytics

Abstract

This systematic review provides an in-depth examination of regression algorithms applied in predictive analytics across multiple domains, including engineering, transportation, and education. The study synthesizes developments in both traditional statistical and modern machine learning (ML) regression models, outlining their evolution, taxonomy, and comparative performance. Traditional multiple linear regression (MLR) has been foundational for predictive analysis but struggles with scalability and computational efficiency in the era of big data. To address these limitations, adaptations such as MapReduce-based MLR have been proposed, enabling distributed computation for large-scale data processing. Simultaneously, hybrid and ML-based regression models—such as support vector regression (SVR), regression trees (CART), and integrated linear–deep learning frameworks—demonstrate superior performance in modeling complex, non-linear relationships.
This review establishes a structured taxonomy differentiating regression approaches based on assumptions, computational properties, and domain-specific requirements, highlighting the balance between model interpretability and predictive power. Comparative evaluations using metrics such as variance explained (R²), mean squared error (MSE), and computational scalability reveal that hybrid regression models often outperform conventional techniques in accuracy and adaptability. The review also identifies persistent challenges related to big data management, model interpretability, data harmonization, and real-time scalability. Emerging research trends emphasize explainable AI (XAI), ensemble regression strategies, and enhanced distributed computing frameworks to overcome these obstacles. Ultimately, this study contributes a holistic understanding of regression-based predictive analytics, guiding researchers and practitioners toward developing scalable, interpretable, and domain-adaptive models for next-generation data-driven decision-making.