Avaliação de desempenho de aglomerado de computadores de placa única para processamento de dados em ambientes IoT usando Apache Spark

Abstract

The constant increase in IoT devices generates large data volumes on networks. As a result, this creates a high demand for processing power and storage. So, many companies have chosen to use cloud servers for data processing due to their high scalability and availability. However, using cloud servers incurs high costs for companies and increases the latency for users accessing their services. To reduce costs and latency, fog or edge computing can be implemented, where data processing is decentralized using multiple servers distributed near the end user, reducing latency and the volume of data sent to cloud servers. However, deploying multiple servers distributed in a specific area becomes costly due to the required hardware and the location to deploy such a system. One solution to reduce the cost of implementing these paradigms is single-board computer clusters, which have a compact size and low cost. Thus, the research goal is to evaluate the performance and energy consumption of single-board computer clusters and compare them with a server machine to determine if their use will bring advantages in processing data generated in IoT environments. A single-board computer cluster composed of different models of Raspberry Pi was created using the Apache Spark framework and evaluated through benchmarks for tasks such as mapping, reducing, and streaming data processing. The results showed that the single-board computer cluster required a longer execution time for data processing than the server machine, resulting in higher total energy consumption than the server machine.