Skip to content

Provides an extensible solution for creating Data Processing Pipelines in F#.

License

Apache-2.0, Apache-2.0 licenses found

Licenses found

Apache-2.0
LICENSE
Apache-2.0
LICENSE.txt
Notifications You must be signed in to change notification settings

marcio-azevedo/fsharp-data-processing-pipeline

Build status NuGet Version NuGet Version

Easily build data processing pipelines in F#

Provides an extensible solution for creating Data Processing Pipelines in F#.


This solution offers the basic infrastructure to build data processing pipelines in F# (or C# due to it's interoperability with F#).

The implementation is based on the Pipes and Filters Pattern. The goal is to provide the basic Infrastructure as well as some External Interfaces implementations to allow an easy and fast setup of a typical Data Processing Pipeline, following Pipes and Filters pattern.

The typical scenario for establishing communication between Pipes and Filters is by providing integration with Queues, so if you have a scenario like this:

You can implement an External Integration with a typical Messaging Queue System and you get a Pub/Sub system (Event-oriented):

The project FSharp.DataProcessingPipelines.Core provides the basic definition of the Core Entities like Messages, Pipes, Filters and Runners.

For more details about how to use / extend this see the Wiki. If you find any issues, please report in Issues.

This is a open source project from and for the community so feel free to contribute (don't forget to send your contact) - just pick a task in Trello Project!

Note: The solution structure is based on the prototypical .NET solution (file system layout and tooling), recommended by the F# Foundation..

About

Provides an extensible solution for creating Data Processing Pipelines in F#.

Topics

Resources

License

Apache-2.0, Apache-2.0 licenses found

Licenses found

Apache-2.0
LICENSE
Apache-2.0
LICENSE.txt

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published