I will audit and optimize your azure data factory and databricks data pipelines

Your pipelines run. Whether they run well is another question. Jobs take longer than they should, costs creep up, reruns create duplicates, and nobody can tell which failures actually matter.

I audit Azure Data Factory and Databricks pipelines and tell you what's broken, what it's costing you, and what to fix first.

What I review:

Pipeline design: orchestration, dependencies, retry and failure handling

Incremental loads: watermarking, idempotency, late-arriving records

Spark performance: partitioning, shuffles, skew, cluster sizing

Delta tables: file sizing, OPTIMIZE/VACUUM, schema evolution

Cost: cluster configuration, DBU spend, idle time

Data quality: validation, reconciliation, logging

You get a written findings report with every issue ranked by impact and effort, specific remediation steps, and code-level recommendations. Not a generic checklist. An actual read of your environment.

Senior Data Engineer, 7+ years building Azure data platforms for enterprise clients. Microsoft Certified: Fabric Analytics Engineer Associate (DP-600).

I also work with Synapse and Microsoft Fabric. Message me with your stack and pipeline count and I'll tell you which package fits.