This project aims to leverage Nepali speech recognition technology to automate home applications, drawing inspiration from popular platforms like Siri and Alexa. The research work is divided into two main components: speech recognition and predictive text implementation into IoT devices/sensors.
The speech recognition aspect utilizes mathematical techniques such as time domain conversion and Short-time Fourier Transform (STFT) for feature extraction. These extracted features are then fed into hybrid CNN-LSTM neural networks for training. The ultimate goal is to convert Nepali speech into predicted text in English language, enabling seamless interaction with home automation systems. This is present in the notebook/model_training.ipynb file
The second part of the project focuses on implementing predictive text functionality into IoT devices/sensors. This is achieved through MQTT (Message Queuing Telemetry Transport) protocols, facilitating bidirectional communications between databases and sensors. By integrating predictive text capabilities, IoT devices become more intuitive and responsive to user inputs, enhancing the overall automation experience. This part is not included here but the general web site implementation for prediotion is shown in the django web app.
This project is based on the research paper titled "Nepali Voice-Controlled Home Automation System", which serves as the foundation for our work.
To use the speech recognition module:
- Ensure the necessary dependencies are installed.
- Train the hybrid CNN-LSTM neural network model using the provided dataset.
- Run the speech recognition script, providing input in Nepali language.
- The script will output the predicted text in English language.