Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Go library to read/write Parquet files. Developed at Twilio Segment

Price Crawler - Tracking Price Inflation

💂‍♂️ Tools to Transform and Query Data with 'Apache' 'Drill'

Query and transform data with PRQL

A library for Spark DataFrame using MinIO Select API

A converter for the OSM PBFs to Parquet files

OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/Parquet dumps

MongoDB integrations for Apache Arrow. Export MongoDB documents to numpy array, parquet files, and pandas dataframes in one line of code.

Library to read a subset of Parquet files

A lightweight Java library that facilitates reading and writing Apache Parquet files without Hadoop dependencies

Threat Detection and Visualization

Converts between file formats such as CSV and Parquet

A web application for viewing Apache Parquet files . This is a Python + Flask application

Streaming data of Tiki with Kafka and processing with Spark, visualize with Elasticsearch & Kibana.

‼️ Handle Big Data for Machine Learning using Python and PySpark, Building ETL Pipelines with PySpark, MongoDB, and Bokeh

💂‍♂️ ☕️ Tools to Transform and Query Data with 'Apache' 'Drill'

A fast and simple command-line (CLI) tool to convert a Parquet file to an Apache Arrow file