Data engineering challenge using Databricks and Apache Spark. The project involved the manipulation of a large dataset from a major tyres market franchise, which required me to use distributed processing with Python notebooks to efficiently analyze and draw insights from the data, using Databricks and Apacbe Spark.
My use of Databricks and distributed processing with Python notebooks allowed me to analyze vast amounts of data quickly and efficiently, providing me with the ability to process and analyze data at scale.
Databricks notebook link: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/1582075949323120/2185557557246094/4945941335205940/latest.html