Enhancing Agile Project Management Education With Ai: Chatgpt-4'S Role In Evaluating Student Contributions

The planning poker estimation technique encour-ages all team members to participate equally, which is essential in the training of future software engineers. By proposing a coordination scheme based on the experience and knowledge of the team members, it enforces the common ownership of effort estimation. Thus, it is crucial that all members contribute to the process [10]. However, given the personal factors that could affect team interaction dynamics, the contributions of team members could not be equally distributed, hindering the goal of the technique. Ensuring the equal participation of team members sets a challenge not only in the professional context but also for training future software developers and team managers [18] that must facilitate team collaboration. Hence, it is vital to detect team members' contributions in order to value collaboration in a development team. In this article, we present the analyses of the interventions of 13 groups of students from the Computer Engineering course at the University of Valparaiso during a user story estimation activity using planning poker. The experimental setup involved computer science undergraduate students, performing a learning activity regarding the Planning Poker estimation technique. The students' interventions were classified according to a human expert following a collaboration framework. Subsequently, they were classified using ChatGPT 4 using the Zero Shot technique in order to compare the automatically generated labels with those provided by human experts. The type of classification used was binary to determine whether or not the intervention analysed was a contribution. The analysis focused on evaluating the accuracy and consistency of ChatGPT in the contribution classification task, considering the model's ability to correctly identify the different types of interventions. The results of this comparison demonstrate the effectiveness of ChatGPT and its potential to assist in real-time evaluation and analysis tasks. This study enhances the understanding of how artificial intelligence tools can complement the work of human experts, improving efficiency and accuracy in educational and agile project management activities.