Yash T

Lead Data Engineer

Software Engineering Daily

Jul 2017 - Jun 2026 • 8 yrs 11 mos

I began my career in data governance with the Government of Gujarat, where my main responsibility was ensuring that official data was accurate, consistent, and compliance with internal standards. This role gave me a strong foundation in data quality management and helped me understand how governance frameworks operate in large organizations. After that, I moved to an educational institute, where I managed databases and handled a major web migration project. Here, I got hands-on experience in data storage management, backup and recovery processes, and user access control. I also began working closely with stakeholders to understand how they use data, which improved my communication and requirement-gathering skills. Next, I joined a core IT company as a Data Engineer, where I spent around four years. This was a turning point in my career because I worked extensively on ETL processes using Microsoft SQL Server, building data integration pipelines to bring in information from multiple systems. Because it was healthcare, we had to comply with HIPAA regulations, so security, privacy, and access control were top priorities. I worked with large and complex datasets, optimized queries for performance, and implemented both full and incremental loads for reporting. I also collaborated directly with clients to understand their needs, troubleshoot issues, and deliver reliable solutions. This role taught me how to handle multi�source data environments and build scalable processes. Currently, I work as a Data Analyst. My role here is more analytics-driven — I scrape data from the web as per specific requirements, clean and process it, and then analyze it to find trends, patterns, and actionable insights. A major project I’m working on right now is an application that integrates data, particularly information about officers, departments, and their movements. In this project, data is our gold mine — accuracy is critical, and I’m responsible for ensuring it’s reliable before it’s use