Development of visibility forecast with elastic lead times using NWP model with DeepAR from past observation for the next 12/24 hours using DeepAR inside sagemaker studio and visualizing using AWS Quicksite with API service for clients for selected airports in the country.
This project aims to forecast the Visibility in the airports (or any parameter) against parameters like wind speed,wind direction,Humidity,Temperature,Dew point temperature.We have used Amazon Sagemaker as the environment for the project which has paved a way to make our cloud based.
- Cloud Based
- Elastic lead times prection from 5 mins to one week
- Decentralized system
- Multivarient Support
- Multi Step predictions
- Platform independant
- Desktop and mobile support
- API Services
- No code / Low code , orchestrated admin pannel
- IOT support with API
- Simple Visuvalizations for the users
- Fast predictions with AWS elastic services
- Real time data feed can be predicted dynamically
- Extensive Database support for uploading real time data
- Architecture
- Development
- Running the App
- Deployment
- Contributing
- Versioning
- Authors
- License
- Acknowledgments
Open and view the Project using the .zip file provided or at my GitHub Repository
The project is also hosted on AWS Platform
This project used AR model, which is Autoregressive model, is Autoregressive (AR) models are a subset of time series models, which can be used to predict future values based on previous observations. AR models use regression techniques and rely on autocorrelation in order to make accurate predictions. Autoregressive model works best with data with short lead times
The performance of progressive predictions of timeseries predictions has been compared against vaerious models like cnnn, rnn ,lstm etc and has shown better accuracy.
It provides an integrated Jupyter authoring notebook instance for easy access to your data sources for exploration and analysis.It also provides common machine learning algorithms that are optimized to run efficiently against extremely large data in a distributed environment.
An integrated machine learning environment where you can build, train, deploy, and analyze your models all in the same application.
- We have centrailised model and output datafile available inside the
S3 Bucketthat can be query by calling the API endpoint created usingAPI GateWay - The Endpoint URL is connected to a
Lambda Function,where we have added a custom Layer to perform data Processing usingPandas
Using DateTime to fetcht the details like visibility,wind speed,wind direction,humidity,dew point temperature and temperature as JSON
-
URL : https://7ytnzu57ti.execute-api.ap-south-1.amazonaws.com/prod/get-data
-
Authorization : None
-
request_body : date_time
-
Connected Lambda Function :
getData
{ "statusCode": 200, "body": "{ "time":"20201101 1400", "Wind_speed":12.0, "Wind_dir":0, "Visibility":2500.0, "Temperature":22.0, "dewpoint":7.0, "Humidity":37.9247722383}" }
Auto Regressor is a Multi Variant model that uses the output of previous iterations as the input of the current iteration which results in better accuracy.
The output from the Auto Regressor model is compared with the observed value in order to check whether the nowcasted data is accurate.
The Nowcasted visibility values are plotted against various factors like temperature. time, dewpoint, and humidity which ensures that the user understands the correlation between the various factors that contribute to visibility.
The predicted data is visualised in the frontend for the user and the data can be also passed using API Keys
<iframe
width="960"
height="720"
src="src_link_here">
</iframe>
<iframe
width="960"
height="720"
src="https://ap-south-1.quicksight.aws.amazon.com/sn/embed/share/accounts/757776451407/dashboards/a647f449-e6c2-4fcc-b7cb-ad6f2e968665?directory_alias=airport-visibility-prediction">
</iframe>
Project Title is open source software [licensed as MIT][license].