I will build databricks pipelines with pyspark

India

I speak English, Hindi

Data Engineer

Data engineer with 5+ years of experience building reliable, well-tested data pipelines on Azure Databricks. I work with Python, SQL, PySpark, Delta Lake and Unity Catalog to build ETL/ELT pipelines f...
About this Gig

Looking for an experienced Databricks data engineer to build a reliable, scalable data pipeline? I'm a Databricks Certified Data Engineer with 5+ years of experience building large-scale ETL pipelines using PySpark, Spark SQL and Delta Lake for global clients in finance and healthcare.


What I deliver:

  • Batch ETL/ELT data pipeline in PySpark and Spark SQL
  • Medallion architecture (Bronze Silver Gold) on Delta Lake
  • Data ingestion from CSV, JSON, Parquet and databases
  • MERGE/upsert, SCD and schema evolution handling
  • Scheduled Databricks Workflows (jobs)
  • Data quality checks and validation
  • Performance-optimized Spark code
  • Architecture diagram and pipeline design notes
  • Clean, well-documented code


Why work with me:

  • Production-grade code based on real indu
  • Experience processing billions of records
  • Clear communication and on-time delivery


Have more than 3 sources or custom requirering and I'll send a custom offer.

Langugae:

English

•

Hindi

Technical expertise:

Apache Spark

•

Databricks

•

Hadoop

Expertise:

Data Pipelines

•

ETL Development

•

Data Lake Setup

Industry:

Data analytics

•

Financial services

•

Food & beverage