Junior Data Engineer
Tata Consultancy Services (TCS) · Jun 2021 – Present
- Designed, built and maintained scalable ETL pipelines using SSIS and SQL for large-scale batch processing, ensuring reliable data movement, improved data quality and timely delivery across enterprise systems.
- Built and maintained ETL/ELT data pipelines using Databricks, PySpark, SQL and Python for large-scale data transformation, validation, reconciliation and production support.
- Implemented Fact and Dimension data models to support centralized data warehouse solutions, integrating data from multiple source systems for business reporting and analytics.
- Optimized SQL queries, indexing and data processing performance, reducing job processing time from 4 hours to 20 minutes.
- Automated data validation, preprocessing and reconciliation with Python (Pandas, NumPy) to ensure data integrity in monthly and yearly business reporting datasets.
- Monitored production ETL workflows and supported migration analysis, resolving issues to keep pipelines reliable.