I have 4+ years of Data Engineering ecosystem. In the past I have developed several end to end scalable data pipelines with log monitoring and orchestration systems to handle billions of data. >> Apache Spark, Spark SQL, PySpark, Dataframe APIs, Databricks Notebook Workflows, Databricks Jobs for ETL big data workloads.
>> .NET data driven applications using C# .NET, ADO.NET, LINQ
>> Design and development of complex ETL and batch data pipelines using MS SQL Server, SSIS, Python
>> Database Administration, query tuning and optimization using Execution plans, IO statistics, Index Optimization etc.
>> Automating repeated tasks in on-prem and Cloud (MS Azure) using Python, PowerShell and Batch Scripting
Apart from work, I code a lot. In my free time, I read technical articles, read books, and maintain technical blogs - you can find my articles on my website.