Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

Our own development branch of the well known WPF document docking library

This repository contains Spark, MLlib, PySpark and Dataframes projects

Azure Databricks - Advent of 2020 Blogposts

Spark-Transformers: Library for exporting Apache Spark MLLIB models to use them in any Java application with no other dependencies.

大数据框架 Spark MLlib 机器学习库基础算法全面讲解,附带齐全的测试文件

Slides, code and more for my class: Data Analytics and Machine Learning on Big Data

Visualizes the Random Forest debug string from the MLLib in Spark using D3.js

Getting started with PySpark for Big data analysis

Basics of Big Data and Machine Learning using Apache Spark and Scala

In this tutorial, I explained SparkContext by using map and filter methods with Lambda functions in Python and created RDD from object and external files, transformations and actions on RDD and pair RDD, PySpark DataFrame from RDD and external files, used sql queries with DataFrames by using Spark SQL, used machine learning with PySpark MLlib.

spark (scala and python)

Example from Spark MLLib (in python)

SparkTDA is a package for Apache Spark providing Topological Data Analysis Functionalities.

dllib is a distributed deep learning library running on Apache Spark

Prediction of Customer Churn using Spark Mllib

A collection of “cookbook-style” scripts for simplifying data engineering and machine learning in Apache Spark.

Random Forest Binary Classification is applying on sample data in PySpark on Jupyter Notebook