I will build, fix, or optimize your python etl data pipeline
Senior Data Engineer and AI Product Engineer
About this Gig
Need a data pipeline built, fixed or improved?
Im a Senior Data Engineer with experience delivering production data systems across banking, construction and startups, including data transformation work on the UBSCredit Suisse integration.
I can help you:
Build ETL/ELT pipelines from APIs, databases or files
Debug and improve existing Python, SQL or PySpark pipelines
Integrate APIs with databases, data lakes or warehouses
Build Databricks, Azure Data Factory or Airflow workflows
Add validation, data quality checks and testing
Improve slow or expensive data processing
Document your pipeline so it is easy to maintain
I work with Python, SQL, Databricks, PySpark, Delta Lake, Azure Data Factory, Airflow, PostgreSQL, SQL Server and Azure data platforms.
Every project is different, so for anything beyond a small fix, please message me before ordering. Ill help define the scope and recommend the simplest solution.
My Portfolio
FAQ
A What do you need from me before starting?
A description of the problem, your current data sources and destination, and any relevant code, schemas or API documentation. For larger projects, please message me before ordering so we can agree the scope.
Can you fix an existing pipeline rather than build a new one?
Yes. I can troubleshoot Python, SQL, PySpark, Databricks, Azure Data Factory and related ETL pipelines, identify the cause of an issue and implement a fix.
Which data sources can you work with?
I can work with REST APIs, SQL databases, PostgreSQL, SQL Server, files such as CSV/JSON/Parquet, data lakes and other common sources. Message me if you have a less common system.
Can you optimise a slow or expensive pipeline?
Yes. I can review the architecture and code, identify bottlenecks and improve areas such as transformations, Spark processing, data layout, parallelism and unnecessary compute.
Can you work with Databricks and Azure?
Yes. Databricks and Azure data engineering are core areas of my professional experience, including PySpark, Delta Lake, Azure Data Factory, ADLS and SQL-based platforms.
Should I contact you before ordering?
For the Basic package, a clearly defined small task can usually be ordered directly. For Standard, Premium or anything where the scope is uncertain, please message me first.
