This repository contains source code based on a guide on how to use Cloud Build and Cloud Composer to create a CI/CD pipeline for building, testing, and deploying a data processing workflow
-
Updated
Jul 12, 2021 - Shell
DataOps is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics. DataOps applies to the entire data lifecycle from data preparation to reporting, and recognizes the interconnected nature of the data analytics team and information technology operations.
This repository contains source code based on a guide on how to use Cloud Build and Cloud Composer to create a CI/CD pipeline for building, testing, and deploying a data processing workflow
Redpanda Console is a developer-friendly UI for managing your Kafka/Redpanda workloads. Console gives you a simple, interactive approach for gaining visibility into your topics, masking data, managing consumer groups, and exploring real-time data with time-travel debugging.
Learn by doing hand-on projects -> https://www.katacoda.com/sntsua
The primary objective of the project is to build a Bash Command Line tool that performs useful data preparation tasks such as cleaning, truncating, and sorting data.
This is a simple DataOps attempt using Liquibase..
gliff.ai [plugin] – a plugin which enables users to access a visual interactive geolocation heat map
Azure Team Data Science Process project template
IntelliJ plugin for editing DataKitchen Platform recipes.
🚀 instant jupyter notebook autolaunch with $HOME mount using docker 🐳
A simple DataOps for wine dataset on Docker
A simple etl pipeline orchestrated with dagster, to run both locally and/or on k8s. Uses minIO as both IOManager and destination when on k8s
DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
Hello there !