SDLC Model For XAI and Program Repair:

Enhancing Software Reliability Through

Explainable AI
Vaibhav Anurag, 221CS258 Rishabh J. Mahor, 221CS229
Computer Science and Engineering Computer Science and Engineering

National Institute of Technology National Institute Of Technology

Karnataka, Surathkal Karnataka, Surathkal

Abstract—The integration of Explainable Artificial Intelligence systems or spend considerable time manually verifying each
(XAI) into program repair holds immense potential to rev- fix, negating the efficiency gains promised by automation.
olutionize software maintenance and development. Traditional
XAI techniques, when applied to program repair, can bridge
automated program repair systems, while effective in identifying
and fixing bugs, often operate as black boxes, making it difficult this gap by providing insights into:
for developers to understand and trust the repairs generated. • The reasoning behind the identification of a particular
XAI addresses this issue by providing insights into the decision-
code segment as buggy
making processes of AI systems. This paper explores the software
development lifecycle (SDLC) for a system that combines XAI • The process of generating and selecting a specific repair
with program repair, aiming to solve the problem of opaque bug • The potential impact of the repair on the overall system
fixes. We provide an in-depth analysis of each SDLC phase, from behavior
requirement analysis to maintenance, and propose a comprehen- • The confidence level of the system in the proposed fix
sive framework for developing such a system. Additionally, we
present case studies to illustrate the practical applications and This paper outlines the SDLC for a system that integrates
benefits of this integration, demonstrating how XAI can enhance XAI into program repair, focusing on addressing the problem
the trustworthiness and efficiency of automated program repair. of opaque bug fixes. We present a detailed analysis of each
SDLC phase and propose a comprehensive framework for
I. I NTRODUCTION developing such a system. The goal is to offer a roadmap
In the realm of software engineering, automated program for creating transparent and effective program repair systems
repair has emerged as a critical tool for maintaining software that leverage the strengths of XAI, ultimately leading to more
reliability and reducing the manual effort required to fix bugs. reliable software and increased developer productivity.
Despite the advancements in this field, traditional automated
program repair tools often lack transparency, rendering the II. P ROBLEM S TATEMENT
generated fixes as inscrutable to developers. This lack of
Automated program repair systems have made significant
explainability can lead to mistrust, hesitancy in adopting auto-
strides in identifying and fixing bugs without human interven-
mated repairs, and, in some cases, the inadvertent acceptance
tion. However, the opacity of these systems presents a major
of flawed fixes.
Explainable Artificial Intelligence (XAI) offers a solution to challenge. Developers often find it difficult to understand the
this challenge by providing mechanisms to make the decision- rationale behind the generated fixes, leading to mistrust and
making processes of AI systems understandable to humans. potential rejection of correct repairs or acceptance of flawed
By integrating XAI into automated program repair, we can ones. This lack of transparency ultimately affects software
create systems that not only fix bugs but also offer clear, reliability and maintainability.
comprehensible explanations for the generated fixes. This The specific problems we aim to address include:
integration aims to enhance the reliability of software and 1) Lack of Trust: Developers are hesitant to accept au-
increase developer confidence in automated solutions. tomated repairs without understanding the reasoning
The importance of this integration cannot be overstated. As behind them.
software systems grow in complexity, the need for efficient and 2) Inefficient Verification: Significant time is spent manu-
reliable bug detection and repair becomes increasingly critical. ally verifying automated fixes, reducing the efficiency
However, the adoption of automated repair tools has been gains of automation.
hindered by the lack of transparency in their decision-making 3) Potential for Introducing New Bugs: Without clear ex-
processes. Developers often find themselves in a position planations, there’s a risk of accepting fixes that may
where they must blindly trust the repairs suggested by these introduce new issues or have unintended consequences.
 4) Limited Learning Opportunities: Opaque fixes don’t • Generate comprehensive reports on bug detection,
contribute to developers’ understanding of the system repair, and system performance.
or improvement of coding practices. • Provide analytics on common bug patterns and
5) Difficulty in Debugging: When automated fixes fail repair strategies.
or cause issues, the lack of explainability makes it
challenging to debug the repair process itself. B. Non-Functional Requirements
The problem we aim to solve is the development of an 1) Scalability:
integrated system that combines XAI with automated program • The system should handle large codebases effi-
repair to provide understandable and trustworthy bug fixes. ciently.
Such a system should offer clear explanations for the proposed • It should support concurrent analysis of multiple
fixes, enhancing developer trust and facilitating the adoption projects or repositories.
of automated program repair solutions. By addressing these 2) Performance:
challenges, we can create a more efficient, reliable, and edu- • Bug detection and repair should be performed
cational approach to software maintenance and development. within acceptable time limits.
• The system should provide real-time feedback for
III. R EQUIREMENTS A NALYSIS small to medium-sized code changes.
A. Functional Requirements 3) Usability:
1) Bug Detection: • Explanations should be easy to understand for de-

• The system should automatically identify bugs in velopers with varying levels of expertise.
the software code. • The user interface should be intuitive and require

• It should support multiple programming languages minimal training.

and paradigms. 4) Security:
• The detection mechanism should be able to iden- • The system should ensure the confidentiality and
tify various types of bugs, including syntax errors, integrity of the code being analyzed.
logical errors, and runtime exceptions. • It should support secure authentication and autho-
2) Bug Repair: rization mechanisms.
• The system should generate fixes for the identified 5) Reliability:
bugs. • The system should maintain high accuracy in bug
• It should prioritize repairs based on bug severity and detection and repair.
potential impact. • It should be robust against crashes and able to
• The repair mechanism should be able to propose recover from failures.
multiple fix options when applicable. 6) Maintainability:
3) Explainability: • The system architecture should be modular to allow
• The system should provide clear explanations for easy updates and extensions.
each proposed fix. • It should include comprehensive documentation for
• Explanations should include the reasoning behind maintenance and future development.
bug identification and repair generation. 7) Compatibility:
• It should offer visualizations to aid in understanding
• The system should be compatible with various op-
complex repairs. erating systems and development environments.
4) User Interface: • It should support different versions of programming
• A user-friendly interface for developers to interact languages and frameworks.
with the system, view bug reports, proposed fixes,
and explanations. IV. D ESIGN
• The interface should allow developers to accept, A. System Architecture
reject, or modify proposed fixes. The system architecture consists of the following compo-
• It should provide options to customize the level of nents:
detail in explanations based on user preferences.
1) Bug Detector:
5) Integration: • Identifies bugs in the source code using static and
• The system should integrate with popular IDEs and dynamic analysis techniques.
version control systems. • Employs machine learning models trained on his-
• It should support API access for integration with torical bug data to enhance detection accuracy.
CI/CD pipelines. • Includes language-specific parsers and analyzers to
6) Reporting: support multiple programming languages.
 2) Repair Generator: • The Explanation Engine analyzes the bug detection
• Proposes fixes for the identified bugs using machine and repair processes.
learning models trained on historical bug-fix data. • It applies XAI techniques to generate explanations

• Utilizes genetic programming and neural network- for why a bug was identified and how the repair was
based approaches to generate diverse repair candi- generated.
dates. • Explanations are tailored based on the complexity

• Incorporates a ranking mechanism to prioritize the of the bug and repair.

most promising fixes. 5) Output:
3) Explanation Engine: • The user interface displays the bugs, proposed fixes,
• Generates human-readable explanations for each and explanations to the developer.
proposed fix using XAI techniques. • Developers can interact with the system, requesting
• Employs techniques such as LIME, SHAP, and more detailed explanations or alternative repairs if
attention mechanisms to provide insights into the needed.
decision-making process. • Feedback from developers is collected and stored
• Includes a natural language generation module for future improvements.
to convert technical explanations into developer- 6) Continuous Learning:
friendly language. • User feedback and repair outcomes are fed back into
4) User Interface: the system.
• Provides a platform for developers to interact with • Machine learning models are periodically retrained
the system. to improve bug detection and repair generation.
• Includes visualization tools to represent code
V. I MPLEMENTATION
changes and their impacts graphically.
• Offers customizable views and explanation levels to A. Bug Detection
cater to different user preferences. Bug detection is implemented using a combination of static
5) Integration Layer: and dynamic analysis techniques. Static analysis involves
• Manages connections with external systems such as examining the code without executing it, using methods such
IDEs and version control systems. as code pattern matching and abstract syntax tree (AST)
• Handles API requests and responses for seamless analysis to identify potential issues. Dynamic analysis, on the
integration with development workflows. other hand, involves monitoring the code’s behavior during
6) Data Management: execution to detect runtime errors and anomalies.
Machine learning models enhance the accuracy of bug
• Stores and manages historical bug data, repair pat-
detection. These models are trained on labeled datasets con-
terns, and user feedback.
taining examples of buggy and non-buggy code. Techniques
• Implements efficient indexing and retrieval mecha-
such as supervised learning, anomaly detection, and deep
nisms for quick access to relevant information.
learning are employed to improve the system’s ability to
B. Data Flow identify bugs accurately.
1) Input: Source code is fed into the system, either through Key implementation aspects include:
direct upload or integration with version control systems. • Static Analysis:

2) Bug Detection: – Abstract Syntax Tree (AST) parsing to analyze code

• The Bug Detector analyzes the code using static and structure
dynamic analysis techniques. – Control flow and data flow analysis
• Machine learning models process the code to iden- – Code smell detection using predefined patterns
tify potential bugs based on learned patterns. • Dynamic Analysis:
• Results are aggregated and prioritized based on – Instrumentation of code for runtime monitoring
severity and confidence levels. – Execution trace analysis
3) Bug Repair: – Memory leak and race condition detection
• The Repair Generator takes the identified bugs as • Machine Learning Models:
input. – Feature extraction from code (e.g., cyclomatic com-
• It generates multiple repair candidates using various plexity, code metrics)
techniques (genetic programming, neural networks, – Training of models using techniques like Random
etc.). Forests, Support Vector Machines, and Deep Neural
• Repair candidates are evaluated and ranked based Networks
on their likelihood of success and potential impact. – Ensemble methods to combine multiple model pre-
4) Explainability: dictions for higher accuracy
B. Repair Generation – Generation of heat maps overlaid on code to indicate
Repair generation involves training machine learning mod- important regions
els on historical bug-fix data. These models use various • Rule-based Explanations:
techniques to propose effective fixes for the identified bugs. – Development of a knowledge base of common bug
Some of the common techniques include: patterns and their fixes
• Genetic Programming: – Implementation of a rule engine to match detected
– Representation of code as genetic sequences bugs with known patterns
– Crossover and mutation operations to generate new – Generation of natural language explanations based
repair candidates on matched rules
– Fitness function based on test case passing and code • Counterfactual Explanations:
similarity – Generation of alternative code versions that would
• Neural Networks: not be flagged as buggy
– Sequence-to-sequence models for learning code – Implementation of minimal edit distance algorithms
transformations to find close non-buggy code
– Attention mechanisms to focus on relevant parts of – Presentation of counterfactuals to explain bug detec-
the code tion boundaries
– Transfer learning from pre-trained code models The Explanation Engine also includes a natural language
• Reinforcement Learning: generation module that converts technical explanations into
– Modeling the repair process as a Markov Decision developer-friendly language. This module uses templates and
Process context-aware text generation to produce clear and concise ex-
– Reward function based on successful compilation planations that can be understood by developers with varying
and test case passing levels of expertise.
– Policy gradient methods for learning optimal repair D. User Interface
strategies
The user interface is designed to be intuitive and user-
• Pattern-based Repair:
friendly, providing developers with a clear view of detected
– Mining of common fix patterns from historical data bugs, proposed fixes, and explanations. Key features of the
– Template-based repair generation user interface implementation include:
– Ranking of generated repairs based on historical
• Code Highlighting:
success rates
– Implementation of syntax highlighting for multiple
C. Explainability programming languages
The Explanation Engine leverages XAI techniques to gener- – Dynamic highlighting of affected code lines to help
ate human-readable explanations for the proposed fixes. Some developers quickly identify bug locations
of the popular XAI techniques used in this context include: – Color-coded highlighting to differentiate between
• LIME (Local Interpretable Model-agnostic Explana- original code, removed code, and added code in fixes
tions): • Interactive Visualizations:

– Implementation of surrogate models to approximate – Implementation of interactive code diff views for
complex repair models locally comparing original and repaired code
– Generation of feature importance scores for code – Integration of force plots and summary plots from
elements contributing to the bug and fix SHAP for visual explanation of feature importance
– Visualization of feature contributions using high- – Development of interactive heat maps for visualizing
lighted code snippets attention weights in neural network-based repairs
• SHAP (SHapley Additive exPlanations): • Explanation Customization:

– Calculation of Shapley values to determine the im- – Implementation of adjustable detail levels for expla-
pact of each code feature on the repair decision nations (e.g., basic, intermediate, advanced)
– Implementation of efficient approximation algo- – Development of filters to focus explanations on spe-
rithms for large codebases cific aspects (e.g., security, performance, style)
– Generation of force plots and summary plots to – Integration of a query system allowing developers to
visualize feature impacts ask specific questions about repairs
• Attention Mechanisms: • Repair Management:

– Integration of attention layers in neural network- – Implementation of a repair acceptance/rejection

based repair models mechanism with reason logging
– Extraction and visualization of attention weights to – Development of a repair history view to track
show model focus changes over time
 – Integration of a collaborative review system for team- • Proper handling of edge cases and error conditions across
based repair decisions component boundaries
• Feedback Mechanisms: • Performance and scalability of the integrated system
– Implementation of rating systems for repair quality • Consistency of explanations with detected bugs and gen-
and explanation clarity erated repairs
– Development of comment threads for discussions on
specific repairs C. User Testing
– Integration of a bug reporting system for issues with User testing involves having developers use the system to
the XAI-PR tool itself identify bugs, apply fixes, and understand explanations. This
• Integration with Development Environments: phase should include:
– Development of plugins for popular IDEs (e.g., VS- • Usability studies with developers of varying experience
Code, IntelliJ) levels
– Implementation of web hooks for integration with • A/B testing of different explanation formats and visual-
version control systems ization techniques
– Creation of API endpoints for programmatic access • Evaluation of developer trust and confidence in the sys-
to the XAI-PR system tem’s recommendations
• Assessment of the system’s impact on developer produc-
VI. T ESTING
tivity and code quality
A. Unit Testing • Collection and analysis of user feedback on the overall
Each component of the system (Bug Detector, Repair Gen- experience
erator, Explanation Engine) should be tested individually to Feedback from user testing should be used to refine the
ensure they function correctly. Unit tests should cover: system’s functionality and usability.
• Bug Detector:

– Accuracy of static analysis on known bug patterns VII. D EPLOYMENT

– Correctness of dynamic analysis in detecting runtime
The system can be deployed as a cloud-based service
errors
or an on-premises solution, depending on the needs of the
– Performance of machine learning models on bench-
organization. Key considerations for deployment include:
mark datasets
• Cloud Deployment:
• Repair Generator:

– Correctness of generated repairs for common bug – Utilization of containerization technologies (e.g.,
types Docker) for easy scaling
– Diversity of repair candidates for complex bugs – Implementation of auto-scaling policies to handle
– Efficiency of repair generation process varying loads
– Setup of secure API gateways for client access
• Explanation Engine:
• On-Premises Deployment:
– Accuracy of LIME and SHAP explanations
– Coherence and readability of generated natural lan- – Development of installation scripts and documenta-
guage explanations tion
– Correctness of visualization generation – Configuration options for integration with existing
development infrastructure
• User Interface:
– Implementation of update mechanisms for model and
– Responsiveness of the UI under various loads system improvements
– Correct rendering of code highlighting and visual-
• Continuous Integration and Deployment (CI/CD):
izations
– Proper handling of user interactions and input vali- – Setup of automated build and test pipelines
dation – Implementation of blue-green deployment strategies
for zero-downtime updates
B. Integration Testing – Integration with monitoring and alerting systems
Integration testing should ensure that the components work • Data Privacy and Security:
together seamlessly and that data flows correctly between – Implementation of data encryption in transit and at
them. Key aspects to test include: rest
• End-to-end bug detection, repair, and explanation gener- – Setup of access control and authentication mecha-
ation process nisms
• Correct propagation of context and metadata between – Compliance with relevant data protection regulations
components (e.g., GDPR)
 VIII. M AINTENANCE C. Explanation
A. Monitoring The Explanation Engine generates an explanation stating
The system should be continuously monitored to ensure it that the proposed fix corrects the formula by aligning it with
performs as expected. Monitoring tools should track metrics the standard financial calculation practices. It highlights the
such as: affected code lines and explains the impact of the fix.
• Bug detection accuracy and false positive rates
The explanation includes:
• Repair success rate and average time to generate repairs • A side-by-side comparison of the original and corrected
• User satisfaction scores and system usage statistics formulas
• System performance metrics (e.g., response time, re- • A mathematical breakdown of how the misplaced paren-
source utilization) thesis affects the calculation
• Error rates and system health indicators • A graph showing the divergence of results between the
incorrect and corrected formulas over time
B. Updates • References to financial calculation standards that support
Regular updates to the machine learning models, explana- the correction
tion techniques, and user interface should be performed to • A natural language summary explaining the importance
improve the system’s performance and keep up with advance- of the fix in layman’s terms
ments in XAI and program repair. This includes:
• Periodic retraining of machine learning models with new D. User Feedback
bug data The developers review the proposed fix and the explanation.
• Integration of new XAI techniques as they become avail- Satisfied with the clarity and rationale, they accept the fix and
able deploy it to production, resolving the critical issue. The team
• UI/UX improvements based on user feedback and usabil- provides positive feedback on the system’s ability to catch
ity studies and explain a subtle but significant bug that might have been
• Performance optimizations and bug fixes overlooked in a manual code review.
• Updates to supported programming languages and frame-
works X. C ONCLUSION
C. Feedback Loop Integrating XAI into program repair offers significant ben-
A feedback loop should be established to collect user efits in terms of transparency, trust, and effectiveness. By
feedback and use it to improve the system continuously. This following a structured SDLC approach, developers can build
can involve: systems that not only fix bugs automatically but also provide
• Regular surveys and user interviews understandable explanations, enhancing software reliability
• Analysis of system usage data and user behavior patterns and developer confidence.
• Tracking of accepted vs. rejected repairs and explanations The proposed system addresses the critical challenge of
• Monitoring of community forums and support channels opaque bug fixes in automated program repair, offering a
• Collaboration with academic researchers and industry solution that combines advanced AI techniques with human-
partners for continuous innovation centric explainability. This integration has the potential to
revolutionize software maintenance practices, leading to:
IX. C ASE S TUDY
• Increased developer trust in automated repairs
A. Scenario • Faster bug resolution times
Consider a software development team using the system • Improved code quality and reliability
to maintain a large codebase for a financial application. A • Enhanced learning opportunities for developers
critical bug is detected that could lead to incorrect financial • More efficient allocation of development resources
calculations. Future work in this area could explore more advanced XAI
techniques, deeper integration with development workflows,
B. Bug Detection and Repair and the application of this approach to other domains of
The Bug Detector identifies the bug related to an incorrect software engineering. As AI continues to play an increasingly
formula in the code. Specifically, it detects a misplaced paren- important role in software development, the principles of
thesis in a compound interest calculation, which could result explainability and transparency will be crucial in ensuring that
in significant errors in long-term financial projections. these tools augment and empower human developers rather
The Repair Generator proposes a fix based on similar bugs than replace them.
in historical data and its understanding of financial formulas. It The journey towards truly intelligent and explainable pro-
suggests moving the closing parenthesis to the correct position gram repair systems is ongoing, and this paper lays the
in the formula. groundwork for future innovations in this exciting field.
 R EFERENCES
[1] S. Wang, T. Liu, and L. Tan, ”Automatically learning semantic features
for defect prediction,” in Proceedings of the 38th International Confer-
ence on Software Engineering, 2016, pp. 297-308.
[2] M. Pradel and K. Sen, ”DeepBugs: A learning approach to name-based
bug detection,” Proceedings of the ACM on Programming Languages,
vol. 2, no. OOPSLA, pp. 1-25, 2018.
[3] R. Just, D. Jalali, and M. D. Ernst, ”Defects4J: A database of existing
faults to enable controlled testing studies for Java programs,” in Pro-
ceedings of the 2014 International Symposium on Software Testing and
Analysis, 2014, pp. 437-440.
[4] M. T. Ribeiro, S. Singh, and C. Guestrin, ””Why Should I Trust You?”:
Explaining the Predictions of Any Classifier,” in Proceedings of the 22nd
ACM SIGKDD International Conference on Knowledge Discovery and
Data Mining, 2016, pp. 1135-1144.
[5] S. M. Lundberg and S.-I. Lee, ”A Unified Approach to Interpreting
Model Predictions,” in Advances in Neural Information Processing
Systems 30, 2017, pp. 4765-4774.
[6] D. Gunning, ”Explainable Artificial Intelligence (XAI),” Defense Ad-
vanced Research Projects Agency (DARPA), nd Web, 2017.
[7] Z. Qi, F. Long, S. Achour, and M. Rinard, ”An Analysis of Patch
Plausibility and Correctness for Generate-and-Validate Patch Generation
Systems,” in Proceedings of the 2015 International Symposium on
Software Testing and Analysis, 2015, pp. 24-36.
[8] R. K. Saha, Y. Lyu, H. Yoshida, and M. R. Prasad, ”ELIXIR: Effective
object-oriented program repair,” in Proceedings of the 32nd IEEE/ACM
International Conference on Automated Software Engineering, 2017, pp.
648-659.
[9] J. Jiang, Y. Xiong, H. Zhang, Q. Gao, and X. Chen, ”Shaping program
repair space with existing patches and similar code,” in Proceedings of
the 27th ACM SIGSOFT International Symposium on Software Testing
and Analysis, 2018, pp. 298-309.
[10] A. Marginean, J. Bader, S. Chandra, M. Harman, Y. Jia, K. Mao, A.
Mols, and A. Scott, ”SapFix: Automated end-to-end repair at scale,” in
Proceedings of the 41st International Conference on Software Engineer-
ing: Software Engineering in Practice, 2019, pp. 269-278.