b
bhamsu007

Shubham

@bhamsu007

Senior Data Engineer

India
English, Hindi
About me
I’m a Data Engineer with 5+ years of experience designing scalable ETL pipelines and cloud data solutions. I work extensively with Azure Data Factory, Databricks, SQL Server, and ADLS to build automated, high‑performance workflows. I also have strong expertise in MSBI, Pentaho, and web‑scraping solutions. My focus is delivering clean architecture, optimized data flows, and clear documentation so businesses can turn raw data into reliable, actionable insights.... Read more

Skills

b
bhamsu007
Shubham
Offline • 

Portfolio

Work experience

Deloitte

Data Engineer

Deloitte • Full-time

Feb 2021 - Present • 5 yrs 5 mos

• Migrated/Modernization of end-to-end data pipelines from SQL Server/MSBI to Azure Data Factory (ADF) and Azure Databricks, ensuring scalability and performance in cloud environments. • Designed and developed data ingestion, transformation, and orchestration pipelines from scratch using ADF, Databricks, ADLS Gen2, and Azure Blob Storage. • Implemented pipeline optimization and performance tuning strategies in ADF and Databricks, resulting in improved reliability and reduced data latency. • Optimized complex SQL stored procedures, achieving up to 40% improvement in execution time and enhanced overall query performance. • Established seamless integration between ADF, Databricks, ADLS Gen2, and SQL Server, ensuring smooth data flow and consistent schema management. • Developed ETL workflows to process data from diverse sources including flat files, REST APIs, and relational databases. • Leveraged Azure Key Vault (secrets) for secure management of credentials and sensitive configuration details in data pipelines. • Collaborated with cross-functional teams to ensure data accuracy, scalability, and cost optimization across Azure resources.