HydroRTC

A Web-Based Data Transfer and Communication Library for Collaborative Data Processing and Sharing in the Hydrological Domain

Table of Contents

• Introduction
• How to Use
• Expansions and Test Cases
• Community
• Feedback
• Scalability and To Do's
• License
• Acknowledgements
• References

Introduction

It is a library to facilitate applications in data sharing and analysis. This library has following features with possible usecases:

How to Use

HydroRTC library is divided into two major components:

1. HydroRTC Client: Corresponding code for this component is in hydrortcclient.js. This component gives access to the functions that are required by the client code of the application to utilize above list of features and usecases.
2. HydroRTC Server: Corresponding code for this component is in hydrortcserver.js. This component manages all the connected peers and performs all the functions that are necessary for the application to utilize above list of features and usecases.

Both of the components communicate with each other using Socket.IO library. HydroRTC Client uses Client API while, HydroRTC uses Server API. In the following figure, Modules, Functions, and data contained inside these components can be seen.

This repository is mainly composed of the following:

1. src: This folder contains all the HydroRTC library files and code.
2. test: This folder contains example for the peer-to-peer and server-to-peer usage of the library.

Deployment

The library is based on NodeJS so you need to install and configure in your environment. Afterwards you can take the next steps to setup the library for publishing or deployment.

The library is dependent on following packages:

1. PeerJS for peer to peer communication using WebRTC.
2. Socket.IO for client-server and server-client communication.

To install these packages, you need to go terminal and point to root folder of the library then type following command to install all the dependencies in package.json.

npm install

After installing the packages, you need to run following command to transpile hydrortcclient.js into a separate build folder to make it available for browser/client code of the application so it imports the required code.

npm run build

Now, the library is ready for distribution.

How to import this library

The hydro-rtc-test is the demo application that demonstrates how to use the HydroRTC. Please note, since the library is based on NodeJS server so, any application that will import this library will become a NodeJS application and all pre-requisites of NodeJS application will be applied.

1. In package.json file of the application please, list the hydro-rtc as dependency as demonstrated in package.json. Please note, this is a package.json file of demo application and its in the same location where library code is placed once, the library is deployed the path of hydro-rtc dependency should be updated accordingly.
2. Then, type the following command in the root folder of the application where package.json file is placed, to download the required dependencies.

npm install

3. After downloading the dependencies, in the server file of the application e-g index.js in case of NodeJS application, import the server component of the library and configure the server accordingly. Please see index.js to get the idea.
4. Then, you need to import the client component of the library in the browser/client code of the application for that purpose you can look at script sections of (index.html)[https://github.com/uihilab/WebRTC/blob/main/hydro-rtc-test/index.html] file. In this file, you can also view how to call appropriate functions to achieve particular feature.
5. Now, the library is imported and this application can be run like any other NodeJS application.

Expansions and Test Cases

• Streaming and consuming large datasets for radar/satellite or sensor data (time series)
  • Streaming Large-scale Data: Visualizing and consuming large-scale streaming datasets as it arrives from server or other peers
    • DEM data
  • Smart Data Transmission: Analysis with first chunks of data and updating the data delivery priorities (start with 15 min, and switch to hourly)
    • Rainfall data streaming at different resolutions (60, 30, 15 minutes)
  • Distributed Data Analysis and Processing: Distributing modeling tasks and receiving results from peers
    • Flood forecasting/mapping - volunteer computing
• Peer to peer data sharing for environmental science - first user downloads data to indexdb and shares this data with second user directly
  • Decentralizing Data Distribution: Peer to peer data sharing (reducing load on the server - decentralizing the data sharing) (1 peer is active - requesting, 2nd peer is passive which is sharing data)
  • Collaborative Data Exchange: Peer to peer model results sharing for collaborative data analysis, communication and exploration (2 peers actively interacting with each other) - chat/file exchange
    • Hydrological analysis, flood forecasting/mapping: Sharing the results with peers
    • VR Multiplayer Synchronization: Data and function sync for remote peers in the same VR room (GeospatialVR)

To test the applications, we have used HDF5, NetCDF, GRIBB, and large TIFF/geoTIFF files that streamed data through dependencies in the node package. This can be made available upon request.

Use case diagrams

1. Sever-to-Peer Data:

2. Smart Data Tranmission:

3. Distributed Data Sharing:

4. Decentralized Data Distribution:

5. Peer to Peer Data Exchange:

Community

The library is meant as a developing environment that allows the deployment of different types of application through an easy to use interface. New workflows and examples can be added by the community of users to exemplofy different examples on how to create web applications for server and client side using the library.

Feedback

Please feel free to send feedback to us on any issues found by filing an issue.

Scalability and To Do's

The framework is not limited to the functions and modules implemented, but rather provides a boilerplate for new features to be added. Nonetheless, the following should be considered:

• The current implementation runs solely on Node.js environment, and thus not optimized to render

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgements

This project is developed by the University of Iowa Hydroinformatics Lab (UIHI Lab):

https://hydroinformatics.uiowa.edu/.

References

• Erazo Ramirez, C., Sermet, Y., Shahid, M., Demir, I. (2024). HydroRTC: A Web-Based Data Transfer and Communication Library for Collaborative Data Processing and Sharing in the Hydrological Domain. Preprint in EarthArxiv. https://doi.org/10.31223/X5PQ5R