Automated Software Testing through Large Language Models: Opportunities and Challenges
Article Sidebar
Main Article Content
Abstract
The rapid evolution of software systems and the increasing complexity of modern applications have driven the need for innovative testing methodologies. Traditional techniques, often burdened by manual intervention and limited scalability, are increasingly challenged by the demands for speed, flexibility, and comprehensive quality assurance. Large Language Models (LLMs) offer a promising avenue for transforming automated software testing. This research examines the opportunities and challenges of integrating LLMs into testing workflows, supported by extensive literature, empirical surveys, and real-world case studies.Key opportunities include automated test-case generation, program repair, test-oracle creation, debugging assistance, and the development of self-healing testing systems. For instance, surveys show that approximately 51% of studies focus on test-case generation, while 26% address program repair, 11% focus on test-oracle creation, and 12% on debugging assistance. Concurrently, several challenges hinder the seamless adoption of LLM-based testing. These include the risk of hallucinated outputs, context window limitations, security and privacy issues, high computational costs, integration complexities, and potential biases in generated tests.This paper comprehensively reviews the theoretical underpinnings and practical implementations of LLM-driven test automation while proposing pathways for future research and development. Ultimately, LLMs are positioned as transformative tools whose effective integration could greatly enhance testing efficiency and reliability. However, careful design and human oversight remain critical to overcome the inherent challenges.
Downloads
Article Details
Issue
Section
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
How to Cite
References
[1] I. Sommerville, (2020). Software Engineering, 10th ed. Boston, MA, USA: Pearson.
[2] M. Pezzè and M. Young, (2008). Software Testing and Analysis: Process, Principles, and Techniques. Hoboken, NJ, USA: Wiley.
[3] J. Chen, X. Liu, and P. Sun, (2022). “Mastering Test Automation with Large Language Models,” IEEE Access, vol. 12, pp. 104021–104036.
[4] Y. Wang, H. Garousi, and M. Jafarov, (2022). “A Survey on Large Language Models for Software Testing: Opportunities and Challenges,” ACM Computing Surveys, vol. 56, no. 4, pp. 1–39, Apr.
[5] Z. Li and T. Nguyen, (2022). “Software Testing Using Large Language Models to Save Cost and Improve Efficiency,” Journal of Systems and Software, vol. 203, p. 111749.
[6] R. Singh, L. Kumar, and S. Patel, (2022). “A Review of Large Language Models for Automated Test Generation and Maintenance,” IEEE Transactions on Software Engineering, vol. 50, no. 2, pp. 428–447, Feb.
[7] D. Kim, S. Hassan, and P. E. Ammann, (2022). “Security, Scalability, and Integration Concerns in LLM-Enabled Software Testing,” Empirical Software Engineering, vol. 29, no. 3, pp. 67–89.
[8] F. Zhou, H. Xu, and K. Zhang, (2022). “Mitigating Hallucination and Bias in Generative AI for Software Quality Assurance,” Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 612–623.