I will build AWS and databricks data pipelines, etl and ai rag apps

I am a data engineer who builds ETL pipeline solutions on Databricks and AWS with PySpark, from ingestion to production platforms.

Senior engineer and tech lead with 4 years of experience. Certified: AWS Certified Data Engineer - Associate and Microsoft Certified Azure Database Administrator Associate.

Selected results:

• Migrated 15+ TB from SQL Server to AWS Aurora PostgreSQL with under 5 minutes downtime using DMS, Glue PySpark and Step Functions
• Azure Databricks lakehouse processing 50M+ records per day: medallion architecture, Delta Lake, Unity Catalog, Azure Data Factory
• Streaming at 2M+ events per day with Kafka/MSK and Spark Structured Streaming
• Spark runtime down 45%, costs down 30%, schema incidents down 90% (80+ pipelines), Airflow failures down 40% (30+ DAGs)

AI solutions:

• RAG chatbots with citations and natural language to SQL tools, shipped as FastAPI apps with a web UI
• Open source on GitHub (OsamaZaidi75): pharma-rag and sql-rag

Project experience includes work for a Fortune 500 pharmaceutical client.

Packages: Data Task Sprint ($200), Pipeline Build ($750), Data Platform / AI App ($1,500). Message me before ordering to confirm scope.