You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Efficient Python data pipeline leveraging Apache Beam and Google Cloud Dataflow to update a Bucket with data concerning daily prices of instruments extracted from BMF website, serving as input for other data pipelines. The code generates a dataflow template, which is then scheduled to run periodically using Cloud Scheduler + Cloud Functions.
Cloud dataflow pipeline code that processes data from a cloud storage bucket, transforms it and stores in Google's highly scalable, reduced latency in-memory database, memorystore which is an implementation of Redis.
Midgard is a wrapper on Beam Kotlin, allowing more concise and expressive code. It removes Beam boilerplate code and proposes more Functional Programming style