Avaliação de atividades de programação submetidas em MOOC com emprego de técnicas de visualização

Abstract

Context: In algorithm or programming classes, it is necessary to create and implement a solution to a problem. However, there is the possibility of having several ways so solve a problem and, consequently, forms of implementations. This, the amount of possible implementations is vast, making it difficult for teachers to evaluate them within reasonable cost, time and quality. Moreover, massive courses, open and online (MOOC) have large numbers of users, aggravating this problem and making it unfeasible individual correction of submissions. Objective: The objective of this project is evaluate and implement supporting tools to assess programming assignments submitted in introductory courses to computing using data mining and visualization techniques to build and show clusters of similar source codes. Method: The first step consisted of identification of features that can be extracted according to static analysis and code style. After this identification, we developed tools to collect such measures so that we could use them for data mining and visualization. Thereby, we improved an existing tool for performing clusters and generating visualizations of submitted programming assingments. To evalute the tool, we have one collection of assignments’ submissions with solutions of 5 different problems. The validation occurred in two steps: through the quality of the visualizations considering the mining techniques and data visualization; and checking the feedback of the tool by the teacher. Results: With a database of 152 implementations, we obtained a good evaluation of the quality of the clusters. Regarding the quality of the visualization for the purpose of evaluating the submitted programs, we conducted an experimental study. The experimental study was based on the use of the ScienceView tool, creating a new database, training the study subjects with respect to the assessment criteria and tool, and assessing the programming submissions. We organized the participants into two groups: one group performed the traditional assessment and the other used the tool to aid in the assessment. These two groups then switched how corrections were made. At the end of the experimental study, the volunteers evaluated the training and the tool positively. Conclusions: Considering the neighborhood preservation, the quality of the projection is compatible with other projections made with the LSP technique and the like, presenting similar results reported in the literature. In relation to the use of visualization for program assessment, the results were limited due to the inefficient use of the tool and the clusters. However, considering clusters containing programs evaluated by the experiment participants, there are indications that visualization can be used successfully and, if we improve training, we can clearly reach our goal. Nevertheless, the participants assessed the use of the tool as positive. As future work, we will investigate the use of other characteristics of the program and the improvement of usability and training regarding the use of the tool.