I will build a python etl pipeline for your data
About this Gig
Have scripts that work but run manually or without proper scheduling?
I'll wrap your existing data workflow in production-grade
orchestration.
What you get:
- Apache Airflow DAGs to schedule and monitor your existing scripts
- Proper error handling and retry logic
- Optional: dbt models to replace ad-hoc SQL transformations with
tested, version-controlled ones
- Optional: data quality tests and source freshness checks
This is the same orchestration pattern I used to run a 1B+ row batch
pipeline reliably scaled down to fit your existing codebase, not
rebuilt from scratch.
Share your current scripts/repo structure in the requirements so I
can scope accurately before starting.
Tools & Platforms:
AWS Glue DataBrew
•
Kafka Connect
•
Other
My Portfolio
FAQ
Do I need to rewrite my scripts?
No — I work with what you have and wrap it in orchestration, minimal changes to your existing logic.
Do you need my scripts before I order?
Not required, but sharing them (or a description of your setup) helps me quote accurately and avoid surprises.
Can you migrate from cron jobs to Airflow?
Yes, this is a common ask and fits well here.
