I will build pyspark and databricks data pipelines and etl

Need a reliable data pipeline, or a Spark job that's too slow or keeps failing? I'm a full-time Data Engineer working with PySpark, Databricks, SQL and Azure every day.

WHAT I CAN DO FOR YOU:

1. Build ETL/ELT pipelines in PySpark and Databricks (CSV, JSON, Parquet, APIs, databases)
2. Load data into Delta Lake, Databricks, Azure Synapse, PostgreSQL or SQL Server
3. Incremental loads, Delta MERGE / upserts, SCD Type 2 history tables
4. Fix slow or failing Spark jobs: data skew, shuffles, joins, small files, out-of-memory errors
5. Data quality checks, deduplication and clean reporting tables
6. Scheduling with Azure Data Factory or Databricks Workflows

WHAT YOU GET:

1. Clean, commented, production-style code
2. A short document explaining how to run and maintain it
3. Before/after runtimes for optimisation work

Please message me before ordering with your data sources, volumes and goal, so I can confirm the right package and timeline. No production credentials needed; sample data or a sandbox is enough to start.