Skip to content
@datacrop

DataCROP

Data Collection ROuting & Processing platform
README.md

DataCROP™ Information

DataCROP™ (Data Collection Routing & Processing) is a Data collection framework which provides the specifications and relevant implementation to enable a real time data collection, transformation, filtering, and management service to facilitate data consumers (i.e., analytic algorithms). The framework can be applied in IoT environments supporting solutions in various domains (e.g., Industrial, Cybersecurity, etch.). For example, the solution may be used to collect security related data (e.g., network, system, solution proprietary, etch.) from monitored IoT systems and store them to detect patterns of abnormal behaviour by applying simple (i.e., filtering and pre-processing) or more elaborated mechanisms (i.e., AI algorithms). The design of the framework is driven by configurability, extensibility, dynamic setup, stream handling capabilities and Blockchain (Ledger) support. One of the key features of the framework is that it is detached from the underlying infrastructure by employing a specialized data model for modelling the solution’s Data Sources, Processors and Results which facilitates the data interoperability discoverability and configurability of the offered solution.

Demonstrator

A DataCROP™ Farro version demo infrastracture instance, which can be deployed locally, can be found at the farro-demo-deployment-scripts repository.

Technologies/Framework

DataCROP has been developed and applied in various iterations, within the context of various EU projects, with different dependencies

DataCROP™ Barley (v1.0) Outcome of FAR-EDGE EU Project

  • MongoDB
  • Apache Kafka (Inter-Data Bus)
  • RabitMQ (Data Input interface)
  • Kafka Streams (Stream Processing)
  • NodeJS (Component Implementation)
  • React (UI)
  • Hyperledger Fabric (optional Blockchain support)

DataCROP™ Farro (v2.0) Outcome of Barley version and PROPHESY EU project

  • MongoDB
  • Apache Kafka (Inter-Data Bus communication & input/output interface)
  • RabbitMQ (Data Input interface)
  • Algorithms: ** Java Algorithms (Qarma) ** Python Algorithms (RUL feature extraction & Health Index Calculation) ** R Algorithms
  • NodeJS (Component Implementation)
  • React (UI)

DataCROP™ Maize 🌽 (V3.0) - (under construction 🚧) Outcome of Farro version and SecureIoT, IoTAC EU projects

  • MongoDB
  • Apache Kafka (Inter-Data Bus)
  • ELK Stack
  • Additional to be included as developments evolves

Maturity Level / Active years

V1.0 - TRL 6 Designed/Developed and demonstrated under the FAR-EDGE (2016-2019) project

V2.0 - TRL6 Designed/Developed and demonstrated under the H2020 PROPHESY (2017-2020) project. Demonstrated under the QU4LITY (2019-2022) project

V3.0 – TRL4 (Under design/Development) Designed/Developed under the SecureIoT (2018-2021) project, IoTAC (2020-2023), STAR (2020-2023)

Future/ Interest Steps

  • Intergrade the design/data models/components into one platform/infrastructure independent solution.
  • Support additional data collection, data processing and data offering services.
  • Offer an intuitive and user-friendly configuration toolbox (UI).
  • Offer data visualization mechanisms (UI).
  • Blockchain (Hyperledger Fabric) integration supporting the configurations and results of the solution.

Links:

Popular repositories

  1. maize-model-repository maize-model-repository Public

    Implementation of databases based on the DataCROP Maize 🌽 (V3.0) digital models.

    Java 2

  2. maize-digital-models maize-digital-models Public

    Digital Models of DataCROP:tm: Maize :corn: (V3.0).

    1

  3. farro-open-api-for-analytics farro-open-api-for-analytics Public

    The DataCROP Farro (V0.2) Open API for Analytics which defines a set of public endpoints that enable external systems to interact with the Distributed Data Analytics component both at the edge and …

    JavaScript

  4. farro-data-router-and-preprocessor farro-data-router-and-preprocessor Public

    Data router and preprocesser is the DataCROP Farro (V0.2) component that enables the edge gateways to interact with field devices.

    JavaScript

  5. farro-mqtt-random-data-publisher farro-mqtt-random-data-publisher Public

    This is a random number generator that publishes the values it generates to an MQTT topic.

    JavaScript

  6. farro-digital-models farro-digital-models Public

    This project contains the DataCROP Farro (V0.2) digital models.

Repositories

Showing 10 of 13 repositories
  • maize-model-repository Public

    Implementation of databases based on the DataCROP Maize 🌽 (V3.0) digital models.

    Java 2 Apache-2.0 0 18 0 Updated Nov 13, 2022
  • .github Public

    Welcome page for the Organization profile and DataCROP framework general info

    0 0 0 0 Updated Nov 3, 2022
  • maize-digital-models Public

    Digital Models of DataCROP:tm: Maize :corn: (V3.0).

    1 Apache-2.0 0 6 0 Updated Oct 29, 2022
  • farro-analytics-processors Public

    Provides a library of simple analytic's processors for the DataCROP Farro (V2.0) Edge Analytics Engine

    Java 0 Apache-2.0 0 0 0 Updated Oct 27, 2022
  • farro-demo-deployment-scripts Public

    DataCROP Farro dockerized platform demo instance deployment

    0 Apache-2.0 0 0 0 Updated Oct 26, 2022
  • farro-mqtt-file-data-publisher Public

    Publishes the values found in a JSON file to an MQTT topic

    JavaScript 0 Apache-2.0 0 0 0 Updated Oct 14, 2022
  • farro-analytics-dashboard Public

    The DataCROP Farro (V0.2) Analytics Dashboard which enables users to interact with the Open API for Analytics

    JavaScript 0 Apache-2.0 0 0 0 Updated Oct 14, 2022
  • farro-edge-analytics-engine Public

    The DataCROP Farro (V0.2) Edge Analytics Engine which enables users to perform analytics tasks "at the edge"

    JavaScript 0 Apache-2.0 0 0 0 Updated Oct 14, 2022
  • farro-model-repository Public

    The DataCROP Farro (V0.2) Model Repository which provides the mechanisms to manage: Data Kinds (DK) Data Interfaces (DI) Data Source Definitions (DSD) Analytics Processor Definitions (APD)

    JavaScript 0 Apache-2.0 0 0 0 Updated Oct 14, 2022
  • farro-digital-models Public

    This project contains the DataCROP Farro (V0.2) digital models.

    0 Apache-2.0 0 0 0 Updated Oct 14, 2022
View all repositories

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…