I will build scalable etl data pipelines using python pyspark and AWS

Saurabh

India

I speak English

2 orders completed

Data Engineer, ETL Pipelines, Databricks, SQL, Azure, AWS

Are you looking for a professional Data Engineer to build robust, scalable ETL pipelines? You're in the right place! 🔹 Python, PySpark, Pandas & SQL 🔹 AWS Glue, S3 & Redshift 🔹 Azure Databricks & A...
Read More
About this Gig

End-to-End ETL Pipelines

  • Design and implement scalable data pipelines using Python, PySpark, AWS Glue, and Databricks for efficient data processing.


AWS Data Engineering

  • Build S3-to-Redshift pipelines with data ingestion, transformation, validation, partitioning, and incremental loading.


Azure Data Engineering

  • Develop scalable solutions using Azure Databricks, ADLS, PySpark, and Delta Lake for modern cloud data platforms.


Data Cleaning & Transformation

  • Use Pandas, PySpark, and SQL to clean, transform, merge, validate, and standardize structured and semi-structured data.


Performance Optimization

  • Optimize Spark and ETL workloads using partitioning, broadcast joins, predicate pushdown, AQE, and optimized Parquet/Delta formats.
Read More
build scalable etl data pipelines using python pyspark and AWS
Full Screen
build scalable etl data pipelines using python pyspark and AWS

Destination Platform:

Amazon Redshift

•

PostgreSQL

•

MySQL

+2 more

Tools & Platforms:

AWS Glue DataBrew

•

Other

Related tags