Um estudo dos padrões de prompt usados em modelos grandes de linguagens durante a realização de tarefas de programação e compreensão de código

Abstract

The popularization of Large Language Models (LLMs) has transformed software engineering, enabling programmers of varying proficiency levels to obtain support in code comprehension and implementation tasks. The primary objective of this research was to understand how different developer profiles interact with these tools, investigating the influence of experience and cognitive style (measured by the GenderMag method) on prompt construction and problem-solving strategies. The methodology consisted of an empirical study involving 27 participants, comprising both students and professionals from various nationalities. Initially, participants completed sociodemographic and cognitive profile questionnaires to be classified into GenderMag personas (Abi and Tim). Subsequently, they performed six development and debugging tasks on a fork of the TodoMVC project within a 40-minute time limit, utilizing GitHub Copilot freely. A mixed-methods data analysis was employed, evaluating quantitative aspects (linguistic metrics and interaction counts) and qualitative aspects (prompt type classification). The results indicated that linguistic effort in prompt formulation is adaptive: more complex tasks result in greater verbosity, although the language remains accessible, with an estimated reading level between the 8th and 9th grades. Qualitatively, the predominant prompt category wasthe "Open Question,"suggesting that users utilize the LLM primarily as a tutor to understand code context rather than merely as a solution generator. Comparative analysis revealed a statistically significant association between the user profile and the type of interaction: TIM profile users tend to be more direct and concise, whereas ABI users are more verbose. Furthermore, experience acted as a moderating factor: ABI novices resorted to excessively long prompts to compensate for a lack of technical vocabulary, while experienced professionals of the same profile overcame expected risk aversion, adopting more objective instructions. It is concluded that interaction with AI-based programming assistants is not uniform, being significantly influenced by the combination of cognitive style and the users’ experience level. This study highlights the need for more inclusive LLM tools capable of assisting in intent synthesis for verbose users and emphasizes the importance of developers enhancing their technical vocabulary for more efficient communication with intelligent agents.