Design and development of a voice assistant for automotive dashboard control

Abstract

To obtain a national driver’s license, Brazilian drivers undergo physical and mental aptitude tests to prove they are qualified to operate a vehicle. However, people who are not in full physical capacity, called Persons with Reduced Mobility, do not have the same freedom to move around due to their limitations. They face difficulties with urban mobility, becoming dependent on public transportation or ride-hailing apps. Another alternative is using specially adapted vehicles, but even these vehicles do not meet all their needs. With the advent of new speech recognition technologies and advanced driver assistance systems, new systems that control the vehicle by voice commands have emerged, acting on multimedia functions or climate control, for example. Thus, this work proposes to develop an embedded system to assist the driver in vehicle conduction using speech recognition and focuses on creating a prototype to evaluate the system’s usability by a Proof of Concept. The V-model of software development was used as the basis of the methodology to create a voice assistant capable of recognizing four commands (right turn signal, left turn signal, hazard warning, and headlights) and controlling the respective functions of the vehicle. The recognition of voice commands was done using a three-step verification that applied artificial intelligence techniques such as neural networks and deep learning. This work also describes the creation of a database of Brazilian Portuguese voice commands for training speech recognition models through Transfer Learning. Besides recognizing the voice commands, the assistant can identify the driver by voice and verify the similarity of the voice command with the driver’s voice. The developed system met all the requirements established in the design stage and correctly recognized 98.4% of the explored cases without noise. In the other cases, no commands were recognized, which is considered better than recognizing another command since this would result in the actuation of an undesired function. Furthermore, the developed prototype was tested in a vehicle in six real driving scenarios, with the sound noise being monitored. The system worked perfectly up to an average of 73.8 dB, which corresponds to the characteristic sound level inside moving vehicles. The processing time for the voice commands was approximately 1 second.