Vehicle industry big data analysis using clustering approaches

Abstract

Working with data has become something fundamental and essential in the modern world. Considering a globalized world economy and industry, data analysis and visualization offer enlightening information for decision making and strategic planning. Data science provides diverse statistical and scientific methods to extract the most value possible from a data set, covering all the preparation, cleaning, aggregation, and manipulation of data. Machine Learning (ML) and Artificial Intelligence (AI) come along with it to learn and explore the data, uncovering things that can not be seen with only the analyst experience. The automotive sector, which significantly influences the world economy and industry, suffers from focusing on technology and short-term focus. Digital transformation has a disruptive effect, considered essential in the sector’s 140 years, causing car companies to offer customized products optimized for customer needs. To work extensively with data and data science becomes fundamental. So, this work brings a study to explore clustering methods in a Big Data dataset of a car company, performing a literature review; treating, normalizing and grouping using the researched methods; and, finally, comparing and analyzing the results. The Knowledge Discovery and Data mining method was used to perform the mining process, comparing the performance of the K-Means, Fuzzy C-Means (FCM) and Self-Organizing Maps (SOM) algorithm through some metrics: sum of squares within clusters (SSW), sum of squares between clusters (SSB), silhouette index (SI) and K-Fold cross-validation with homogeneity score. When evaluating the vehicle’s distribution of the results obtained, the ML algorithms tend to distribute more evenly among the clusters than the classification without learning, and the SI metric proves this as a good decision. The methods brought to this work showed satisfactory results on the dataset, and demonstrate how the application of ML can bring benefits to data mining. With this, we managed to answer the question "How can historical usage data help a truck manufacturer improve product development and fuel consumption?". K-Means is a good and main clustering technique, while FCM has also proved to be a good technique, working mainly with overlapping situations. FCM also brings an extra interpretation of cluster membership percentage that can help end users understand the data even more. For future works, it can be also implemented K-Medoids as an alternative method that considers an individual as the center of the cluster. This work can also be extended to other types of vehicle’s data set.