I will build etl pipelines using pyspark, azure data factory, data bricks

Hello!

I am an experienced Data Engineer specializing in modern data technologies. With hands-on expertise in PySpark, Azure Data Factory, Databricks, Python, and SQL, I deliver tailored solutions that ensure data is efficiently processed, moved, and transformed.

What I Offer:

PySpark Development:

• Data processing, transformation, and analytics
• Spark job optimization for better performance

Azure Data Factory (ADF):

• Building and scheduling data pipelines
• Connecting multiple data sources and destinations

Databricks Services:

• Managing and optimizing Spark clusters
• Interactive notebooks for data transformation and analysis

Python Scripting:

• Writing scripts for automation and data manipulation
• Integrating Python with ADF or Spark workflows

SQL Expertise:

• Query optimization and performance tuning
• Database schema design and complex joins