(10) Agile_Project_Management_Using_Large_Language_Models
(10) Agile_Project_Management_Using_Large_Language_Models
Models
Dhruva G Ishaan Shettigar Srikrshna Parthasarthy
Department of Computer Science & Department of Computer Science & Department of Computer Science &
Engineering Engineering Engineering
2024 5th International Conference on Innovative Trends in Information Technology (ICITIIT) | 979-8-3503-8681-3/24/$31.00 ©2024 IEEE | DOI: 10.1109/ICITIIT61487.2024.10580873
Sapna V M
Department of Computer Science &
Engineering
PES University
Bengaluru, India
sapnavm@pes.edu
Authorized licensed use limited to: Centre de Recherches et des Technologies des Eaux. Downloaded on July 31,2024 at 07:15:17 UTC from IEEE Xplore. Restrictions apply.
B. Agile Techniques
The Agile philosophy encompasses more than a dozen
techniques within the software development lifecycle,
allowing enterprises to tailor their implementation according
to their specific requirements and the nature of their work.
Some of these Agile practices have been elaborated below.
• Scrum – Scrum is a prominent agile project
management framework characterized by iterative
and incremental development methodologies.
Within the Scrum framework, interdisciplinary Fig. 2. A variant of the Kanban Board
teams engage in brief, time-bound sprints with the
objective of delivering a potentially deployable III. CONTEMPORARY CHALLENGES
product increment at the culmination of each Despite the remarkable leapfrogging in software
iteration. The Scrum teams are characterized by a development achieved with Agile practices, the existing
Product Owner and Scrum Master, who decide framework has continued to lag behind a vigorously evolving
Sprint Planning, Daily Scrum, Product Backlogs and technological landscape, threatening the philosophy with
Sprint Backlogs [5]. obsolescence in the absence of innovation. In this context, it
• XP – Key principles of XP (eXtreme Programming) becomes necessary to outline the contemporary challenges [9]
include pair programming, frequent code integration in the effective implementation of the Agile principles, paving
and testing, and test-driven development to ensure the way for a systematic enhancement and upgradation of the
rigorous development and testing practices. aforementioned practices.
Additionally, XP mandates collective code As organizations expand globally, the workforce has
ownership coupled with disciplined development increasingly become disparate and cross-national, enhancing
practices, making it efficient for large-scale projects the challenges in managing cycles of iterative software
with nebulous requirements resulting in frequent development across time zones and nationalities.
collaboration with customers. XP continues to thrive Organizations rely on sub-optimal Agile project management
as it is a lightweight and highly flexible development practices, often attributable to the lack of trained Agile
framework [6]. practitioners within the organization. Furthermore, the
• Feature-Driven Development – FDD involves a cultural shift required makes the adoption of Agile processes
structured five-step process, comprising the resource-consuming. Furthermore, the absence of data points
development of an overarching model, delineation of and feedback loops make project management inherently
a feature list, planning, designing, and building heuristic, making it exceedingly vulnerable to human errors,
features iteratively. Central to FDD is the concept of inaccurate project estimates and insufficient resource
features, denoting discrete, client-visible allocation. These compounds significantly in the downstream
functionalities prioritized based on business value. processes.
This methodology fosters a refinement-based The need for constant collaboration with customers or
development framework based on a client-centric clients may significantly impede the pace of development due
approach, ensuring efficient collaboration between to ambiguous requirements or infrequent interactions. Some
the development team and stakeholders [7]. of these impediments include lack of communication among
C. Kanban stakeholders, deferred replies and deferred communication,
unavailability of customers and frequently evolving
The genesis of Kanban can be traced back to the late 1940s requirements from different stakeholders [10].
when Taiichi Ohno, a key architect of the Toyota Production
System (TPS), developed Kanban as a visual scheduling and In an environment where consumer behaviour is dictated
production control system to optimize manufacturing by the responsiveness, agility and turnaround time of
processes. The term "Kanban" (看板) in Japanese translates to enterprises, it becomes essential to evolve contemporary
"visual card." The tasks are represented as tiles on a Kanban strategies and techniques to address the needs of a fiercely
board and this traces the task across the project lifecycle. As competitive market. This can be accelerated by leveraging
the task progresses through the development process, the tiles emerging technologies and management philosophies to
are accordingly moved across different phases on the Kanban mitigate the roadblocks prevalent in the current Agile
board (as shown in Figure 2). processes.
In the present milieu, the utilization of Kanban aims to IV. RELATED WORKS
enhance transparency throughout the software development
lifecycle, and it serves as an efficient methodology for A. Literature Survey
implementing Agile and DevOps principles [8]. The Bharat Choudhary and Shanu K Rakesh (2016) [11]
aforementioned framework emphasizes the need for attempt to coalesce the diverging techniques used in Agile
instantaneous communication of capacity and the unhindered project management, and elaborately discuss the
visibility of work progress. Visual representation using tiles methodologies adopted by respondents. Their research
on the Kanban board empowers cross-functional team outlines Scrum (evident in Figure 3) as the most popular Agile
members to glean real-time insights into the status of each framework. The authors further enumerate the different
work item, instilling a heightened level of transparency techniques, advantages and limitations of Adaptive Software
throughout the software development process. Development (ASD).
Authorized licensed use limited to: Centre de Recherches et des Technologies des Eaux. Downloaded on July 31,2024 at 07:15:17 UTC from IEEE Xplore. Restrictions apply.
Sergey Bushuyev et al. (2022) [14] introduce a novel
metric for assessing the quality of project management
processes using entropy. Project management quality is
evaluated through the information project entropy,
representing the team's confidence in project outcomes. A
management quality scale and the entropy index are proposed
for continuous monitoring throughout the project life cycle, as
illustrated in Figure 5. Three pathways of project
informational entropy are explored, emphasizing the ideal
scenario representing a consistent decrease in entropy as the
project lifecycle progresses. This research proposes the use of
quantifiable metrics to evaluate project management
Fig. 3. 65% of the respondents used Scrum (N=125) practices.
Authorized licensed use limited to: Centre de Recherches et des Technologies des Eaux. Downloaded on July 31,2024 at 07:15:17 UTC from IEEE Xplore. Restrictions apply.
V. GENERATIVE AI & LARGE LANGUAGE MODELS VI. PROPOSED METHODOLGY
A. Introduction A. Architecture
The rapid emergence and disruptive popularity of General Figure 8 introduces an Agile project management
Artificial Intelligence (GAI) has ushered in a new paradigm of architecture using the PaLM large language model coupled
efficient computing [16]. While it remains early days for the with internal enterprise data.
advent of GAI, it has quickly superseded traditional
computational techniques, percolating to several industries.
Unlike traditional machine learning algorithms trained to
perform exceptionally well for specific tasks, GAI attempts to
use large corpus of data to model realistic environments, with
the ability of the model to replicate or accelerate human
behaviour to a remarkable extent in several tasks.
A Large Language Model (LLM) is a statistical deep
learning model that has undergone extensive training on a vast
multimodal dataset, and it possesses the capability to generate
text, generate multimedia content, assist in summarization and
perform several challenging natural language processing
(NLP) tasks. Such models are trained on several thousand
gigabytes of multimodal data, to facilitate the development of Fig. 8. Intelligent Agile project management using LLMs
a general artificial intelligence.
The premise of an omni-channel interface facilitates
While safety and accuracy of such LLMs need further customers and end users raising tickets, feature requests or
quantification and research, the models undergo continuous other requirements on an ad hoc basis. These are stored
fine-tuning and refinement based on a robust feedback loop, internally on a cloud-based MongoDB Atlas database for
enabling the model to stay relevant in the face of constant further processing and refinement.
disruption and emergence of newer data. Industries continue
to explore and leverage such large language models to A large language model (PaLM in this instance) is used to
automate several routine tasks. This paper attempts to explore augment the information obtained from the clients, end-users
the viability of using such large language models to accelerate or customers using a large corpus of training data. A single
Agile project management of large-scale software projects. query is augmented to include the following fields – task
name, description, teams capable of resolving the ticket raised
B. PaLM LLM and the priority. The information obtained is used to forecast
Pathways Language Model (PaLM) is a dense decoder- potential deadlines for the new ticket, enabling the project
only Transformer model trained on 540-billion parameters. manager or Scrum master to leverage internal enterprise data
PaLM uses the Pathways system [17] that adeptly orchestrates to estimate the potential endpoint, enhancing the capabilities
diverse parallel computations across thousands of accelerators of existing enterprise-agnostic project management processes.
through efficient scheduling, while coordinating seamless The proposed intelligent task augmentation and allocation
data transfers. Through extensive evaluations on numerous methodology delves into relevant enterprise-specific details
language understanding and generation tasks, PaLM such as historical project timelines and resource allocation
consistently demonstrates state-of-the-art few-shot patterns. Such holistic perspective empowers Agile project
performance, surpassing existing benchmarks by significant managers to take informed decisions on task scheduling,
margins. Figure 7 [18] shows the performance of PaLM on optimize resource utilization, and mitigate risks effectively
scaling, benchmarked on the “Beyond the Imitation Game” [19].
scale against other LLMs. Additionally, PaLM performs well
on multilingual NLP benchmarks, such as translation and text B. Synthetic Dataset
summarization. Due to the absence of a publicly available enterprise
dataset, we were compelled to create a synthetic dataset [20]
to simulate the necessary intelligent task allocation. Attributes
such as the number of personnel allocated and priority were
used to understand their correlation with the duration of the
ticket raised on the Kanban board. Figure 9 is a sample of the
dataset used.
Fig. 9. Synthetic dataset used for estimating the duration of a new ticket
Fig. 7. Scaling behavior of PaLM on a subset of 58 BIG-bench tasks.
The dataset was synthetically created by using generative
This paper introduces the use of PaLM to facilitate AI (i.e., ChatGPT), where a list of 250 tickets with their
automated Agile task creation and management and explores necessary attributes was generated along with their duration.
the performance of the model in this context. These were validated by a project management professional to
ensure that the synthetic dataset reflected the ground truth.
Authorized licensed use limited to: Centre de Recherches et des Technologies des Eaux. Downloaded on July 31,2024 at 07:15:17 UTC from IEEE Xplore. Restrictions apply.
VII. RESULTS & DISCUSSION TABLE I. ACCURACY METRICS
The analysis of the results is two pronged, making it Sl. No. Regression Model MSE R2 Score
imperative to delve into the inferences of the results obtained. 1 Linear Regression 1.234 0.7433
These inferences have guided downstream design and
implementation choices. 2 Polynomial Regression 0.8492 0.8234
Priority High
Authorized licensed use limited to: Centre de Recherches et des Technologies des Eaux. Downloaded on July 31,2024 at 07:15:17 UTC from IEEE Xplore. Restrictions apply.
VIII. CONCLUSION & FUTURE WORK [10] A. K. Nisyak, K. Rizkiyah and T. Raharjo, "Human Related Challenges
in Agile Software Development of Government Outsourcing Project,"
This paper attempts to identify the potential of LLMs for 2020 7th International Conference on Electrical Engineering,
automating Agile project management. This delves into a new Computer Sciences and Informatics (EECSI), Yogyakarta, Indonesia,
paradigm of data-driven Agile task scheduling and opens up 2020, pp. 222-229, doi: 10.23919/EECSI50503.2020.9251899.
new avenues of innovation in software management. [11] B. Choudhary and S. K. Rakesh, "An approach using agile method for
software development," 2016 International Conference on Innovation
Furthermore, the research introduces a novel coupling of large and Challenges in Cyber Security (ICICCS-INBUSH), Greater Noida,
language models with enterprise data. In this context, several India, 2016, pp. 155-158, doi: 10.1109/ICICCS.2016.7542304.
regression techniques for heuristic task scheduling have been [12] A. Ahmed, S. Ahmad, N. Ehsan, E. Mirza and S. Z. Sarwar, "Agile
elaborately discussed. software development: Impact on productivity and quality," 2010 IEEE
International Conference on Management of Innovation &
Future research on intelligent task-based scheduling and Technology, Singapore, 2010, pp. 287-291, doi:
reinforcement-based task management can usher in a new era 10.1109/ICMIT.2010.5492703.
of Agile project management. Furthermore, LLMs can be [13] N. Saleem, S. Mathrani and A. Sajjad, "Exploring the Evolutionary
effectively employed for analyzing screenshots uploaded by Characteristics of Project Management Approaches at Different Levels
customers for problem resolution. The next decade beacons a of Operations," 2021 IEEE Asia-Pacific Conference on Computer
Science and Data Engineering (CSDE), Brisbane, Australia, 2021, pp.
monumental disruption in traditional project management 1-7, doi: 10.1109/CSDE53843.2021.9718471.
processes.
[14] S. Bushuyev, S. Onyshchenko, A. Bondar, N. Bushuyeva and A.
Obronova, "Assessment and Monitoring Project Management Quality
REFERENCES Based on the Entropy Approach," 2022 International Conference on
[1] A. Ahmed, S. Ahmad, N. Ehsan, E. Mirza and S. Z. Sarwar, "Agile Smart Information Systems and Technologies (SIST), Nur-Sultan,
software development: Impact on productivity and quality," 2010 IEEE Kazakhstan, 2022, pp. 1-5, doi: 10.1109/SIST54437.2022.9945775.
International Conference on Management of Innovation & [15] S. Zhang and L. Jin, "Research on Software Project Schedule
Technology, Singapore, 2010, pp. 287-291, doi: Management Method based on Monte Carlo Simulation," 2020 IEEE
10.1109/ICMIT.2010.5492703. 5th Information Technology and Mechatronics Engineering
[2] B. Choudhary and S. K. Rakesh, "An approach using agile method for Conference (ITOEC), Chongqing, China, 2020, pp. 1605-1608, doi:
software development," 2016 International Conference on Innovation 10.1109/ITOEC49072.2020.9141570.
and Challenges in Cyber Security (ICICCS-INBUSH), Greater Noida, [16] M. Jovanović and M. Campbell, "Generative Artificial Intelligence:
India, 2016, pp. 155-158, doi: 10.1109/ICICCS.2016.7542304. Trends and Prospects," in Computer, vol. 55, no. 10, pp. 107-112, Oct.
[3] F. Paetsch, A. Eberlein and F. Maurer, "Requirements engineering and 2022, doi: 10.1109/MC.2022.3192720.
agile software development," WET ICE 2003. Proceedings. Twelfth [17] P. Barham, A. Chowdhery, J. Dean, S. Ghemawat, S. Hand, D. Hurt,
IEEE International Workshops on Enabling Technologies: M. Isard, H. Lim, R. Pang, S. Roy, B. Saeta, P. Schuh, R. Sepassi, L.
Infrastructure for Collaborative Enterprises, 2003., Linz, Austria, 2003, El Shafey, C. A. Thekkath, and Y. Wu, "Pathways: Asynchronous
pp. 308-313, doi: 10.1109/ENABL.2003.1231428. Distributed Dataflow for ML," 2022. Available:
[4] P. Sankhe, S. Mathur, T. B. Rehman and M. Dixit, "Review of an Agile https://arxiv.org/abs/2203.12533.
Software Development Methodology with SCRUM & Extreme [18] S. Narang and A. Chowdhery, “Pathways Language Model (PaLM):
Programming," 2022 IEEE International Conference on Current Scaling to 540 Billion Parameters for Breakthrough Performance,”
Development in Engineering and Technology (CCET), Bhopal, India, https://blog.research.google/2022/04/pathways-language-model-palm-
2022, pp. 1-6, doi: 10.1109/CCET56606.2022.10080640. scaling-to.html (accessed Jan. 4, 2024)
[5] A. Srivastava, S. Bhardwaj and S. Saraswat, "SCRUM model for agile [19] H. Wang, Z. Ren, Z. Yu, Y. Zhang, J. Liu and H. Cui, "CoupHM: Task
methodology," 2017 International Conference on Computing, Scheduling Using Gradient Based Optimization for Human-Machine
Communication and Automation (ICCCA), Greater Noida, India, 2017, Computing Systems," 2022 IEEE 28th International Conference on
pp. 864-869, doi: 10.1109/CCAA.2017.8229928. Parallel and Distributed Systems (ICPADS), Nanjing, China, 2023, pp.
[6] R. Juric, "Extreme programming and its development practices," ITI 859-866, doi: 10.1109/ICPADS56603.2022.00116.
2000. Proceedings of the 22nd International Conference on Information [20] M. Abufadda and K. Mansour, "A Survey of Synthetic Data Generation
Technology Interfaces (Cat. No.00EX411), Pula, Croatia, 2000, pp. 97- for Machine Learning," 2021 22nd International Arab Conference on
104. Information Technology (ACIT), Muscat, Oman, 2021, pp. 1-7, doi:
[7] S. Thakur and H. Singh, "FDRD: Feature driven reuse development 10.1109/ACIT53391.2021.9677302.
process model," 2014 IEEE International Conference on Advanced [21] D. Chicco, M. J. Warren, G. Jurman, "The coefficient of determination
Communications, Control and Computing Technologies, R-squared is more informative than SMAPE, MAE, MAPE, MSE and
Ramanathapuram, India, 2014, pp. 1593-1598, doi: RMSE in regression analysis evaluation," PeerJ Comput Sci., vol. 7,
10.1109/ICACCCT.2014.7019376. e623, Jul 5, 2021. DOI: 10.7717/peerj-cs.623. PMID: 34307865;
[8] M. O. Ahmad, J. Markkula and M. Oivo, "Kanban in software PMCID: PMC8279135.
development: A systematic literature review," 2013 39th Euromicro [22] A. M. A. Sai, K. L. Sai Eswar, K. S. Sai Harshith, P. Raghavendra, G.
Conference on Software Engineering and Advanced Applications, Y. Kiran and M. V., "Study of Lasso and Ridge Regression using
Santander, Spain, 2013, pp. 9-16, doi: 10.1109/SEAA.2013.28. ADMM," 2022 2nd International Conference on Intelligent
[9] W. R. Fitriani, P. Rahayu and D. I. Sensuse, "Challenges in agile Technologies (CONIT), Hubli, India, 2022, pp. 1-8, doi:
software development: A systematic literature review," 2016 10.1109/CONIT55038.2022.9847706.
International Conference on Advanced Computer Science and
Information Systems (ICACSIS), Malang, Indonesia, 2016, pp. 155-
164, doi: 10.1109/ICACSIS.2016.7872736.
Authorized licensed use limited to: Centre de Recherches et des Technologies des Eaux. Downloaded on July 31,2024 at 07:15:17 UTC from IEEE Xplore. Restrictions apply.