Browse categories
Explore
Fiverr Pro
English
$
USD
I will be your expert databricks and pyspark data engineer for etl pipelines
Senior Data and AI Engineer, Databricks Certified, 11 years
I am an experienced professional with 11+ years in software and data engineering. I have a proven track record leading agile teams, automating ETL processes and infrastructure, optimizing data pipelin...
About this Gig
Need bulletproof Databricks pipelines that don't break at 3am? You're in the right place.
I'm a Databricks Certified Data Engineer (Professional) with 10+ years building production data systems for energy trading, banking, retail, and crypto-analytics companies including OVO Energy, Bank of America, and Sainsbury's.
What I'll build for you:
- ️Scalable PySpark / Scala Spark pipelines on Databricks
- DeltaLake architecture (bronze/silver/gold medallion)
- Streaming + batch ETL with Kafka, Auto Loader, or Structured Streaming
- Cost-optimized cluster configurations (I've delivered £80k+ in annual cloud savings)
- Clean, tested, documented code you can hand to your team
My stack: Databricks, PySpark, Scala, DeltaLake, Airflow, Kafka, Snowflake, BigQuery, Terraform, AWS, GCP.
My process: Discovery call -> architecture sign-off -> build -> test -> walk-through -> handover. No surprises, no scope creep.
Message me before ordering so I can scope your project properly and confirm fit. I respond within 1 hour during UK business hours.
Let's build something solid.
Expertise:
API integration
•
Automations
•
Big data
•
Data flow
•
ETL
•
SQL
FAQ
What information do you need to get started?
Source data location and format, target destination, sample data (anonymized is fine), volume estimates, and SLA requirements. I'll send a short brief template after order.
Can you work with my existing cloud setup?
Yes — I work with AWS, GCP, and Azure-hosted Databricks. I'll work inside your environment using read-only credentials you provision.
Do you sign NDAs?
Absolutely. Send me your NDA before ordering and I'll review and sign.
What if I don't have data yet?
No problem — I can mock realistic data for development and you can swap in production data later.
Do you handle real-time streaming pipelines?
Yes — Structured Streaming, Auto Loader, and Kafka integrations are core strengths.
What happens after delivery?
You get a walkthrough (call or Loom), all source code, README, and 14 days of email support for questions. Premium tier includes a 30-min handover call.
What support do I get after delivery?
Every order includes 14 days of email support — I answer questions within 24 hours, and if something I built isn't behaving as specified, I'll fix it fast at no extra charge. The Premium package adds a 30-minute handover call so you and your team are fully comfortable running and extending the work.
