Método baseado em inteligência artificial para a avaliação e melhoria da qualidade do product backlog em ambientes ágeis em escala: uma abordagem baseada em Design Science Research

Abstract

Managing the quality of the Product Backlog represents a recurring challenge in scaled agile environments, where multiple teams simultaneously develop complex and continuously evolving products. From a Production Engineering perspective, the backlog can be understood as the primary informational input of the software development process, playing a role analogous to that of raw materials in traditional production systems. When this input contains ambiguities, inconsistencies, or low informational quality, the development process tends to exhibit rework, increased variability, and reduced delivery predictability. Despite its importance, backlog evaluation is often conducted in a subjective manner, with limited standardization and low scalability, which compromises the maturity of planning processes and the governance of software development in complex organizational environments. In this context, this dissertation aims to propose, implement, and evaluate a structured method for assessing and improving the quality of the Product Backlog supported by Artificial Intelligence. The research adopts the Design Science Research (DSR) approach, involving problem identification, artifact design, implementation, and evaluation within a real organizational context. The proposed method is structured as an integrated process composed of three complementary components: (i) systematic evaluation of backlog item quality, (ii) support for writing and continuously improving requirement descriptions, and (iii) continuous monitoring of the agile planning process. The evaluation framework is based on four central dimensions of agile requirements engineering: user story description, acceptance criteria, Definition of Ready, and Definition of Done. For each dimension, explicit evaluation criteria and an ordinal measurement scale are defined, enabling qualitative assessments to be transformed into structured quality indicators. The method is operationalized through a computational artifact based on LLM capable of automatically analyzing textual backlog artifacts, generating structured diagnostics, and providing improvement recommendations. The method also incorporates periodic collective evaluation cycles and visual management mechanisms aimed at promoting organizational learning and stimulating the continuous improvement of requirement quality. The application of the method in an organizational environment adopting scaled agile methodologies demonstrated its technical and operational feasibility. The results indicate that the solution contributes to reducing evaluation subjectivity, increasing the scalability of backlog analysis, and supporting the systematic identification of recurring weaknesses in requirements. Therefore, the proposed artifact contributes to strengthening agile planning governance and improving the informational quality that supports large-scale software development, reinforcing classical Production Engineering principles related to quality at the source, process standardization, and continuous improvement.