
Bhagath J
Lead Data Engineer
Skills

See my services

Work experience
Lead - technology (Data Engineer)
Company • Full-time
May 2026 - Present • 4 mos
● Designing and implementing PySpark/Python notebooks in Microsoft Fabric to author, configure, and execute data quality validation workflows. ● Building a QC Authoring Suite and interactive QC Playground Notebook enabling SMEs to manage domain-specific Great Expectations (GX) rules and automate configuration JSON file generation. ● Developing a QC Execution Notebook to execute validation rules against large-scale datasets in Fabric Lakehouse, producing automated run outputs and exception data. ● Automating exception detection using PySpark/SQL and integrated results into KQL Validation tables and RTM tools, streamlining issue resolution for business teams. ● Utilized GitLab (DXOne) for enterprise version control, merge requests, and audit compliance.
Senior Data Engineer L2
Publicis Sapient • Full-time
Apr 2025 - Apr 2026 • 1 yr
● Migrated 11+ legacy ETL workflows from Informatica/SSIS to Airflow, ADF, ADLS, and Snowflake, cutting infrastructure costs by ~40%. ● Orchestrated 15+ Airflow pipelines and PySpark jobs on Azure Databricks to process 1.5 TB of transactional data, reducing job runtime by 60% and cluster costs by 30%. ● Architected raw and processed landing zones in ADLS Gen2 with validation frameworks, achieving 100% audit compliance and reducing data discrepancies by 98%. ● Tuned Snowflake performance through SQL query optimization, lowering compute costs by 35% and improving load times by 55%. ● Integrated 15+ new business rules into transformation logic, improving data quality scores by 25%. ● Authored technical SOPs and migration blueprints, reducing onboarding ramp-up time for new engineers by 50%.
Freelancing
Company • Freelance
Jan 2026 - Jan 2026 • 0 mos
Role: Freelance BI Developer Duration: 4 weeks Project Focus: Designed and developed interactive dashboards in Power BI tailored to client reporting needs. Connected Power BI to multiple data sources (Excel, SQL databases, and cloud storage). Built data models with relationships, measures, and calculated columns for accurate reporting. Implemented DAX expressions to enable advanced analytics (KPIs, trend analysis, and custom metrics). Applied data cleansing and transformation using Power Query to ensure consistency and reliability. Optimized reports for performance and usability, including role‑based access and drill‑through features. Impact: Delivered a complete reporting solution that improved visibility into business operations, reduced manual reporting effort, and enabled stakeholders to make faster, data‑driven decisions.