Divyansh Mittal

With nine years as a Systems Engineer, I am driven by an ambitious curiosity to master complex challenges, always seeking to build robust, future-proof solutions. My approach is rooted in logical analysis and a flexible mindset, allowing me to adapt and innovate while consistently striving for excellence and continuous improvement.

Experience

Data Engineer

Tata Consultancy Services · Jun 2021 – Present

Not yet confirmed

• Retail Data & Insights Platform
• • Led the development of a scalable Retail Data Lakehouse platform for a global FMCG enterprise using Azure
• Data Factory, Databricks, ADLS Gen2, and Delta Lake, seamlessly processing 3M+ sales records daily.
• • Built reusable PySpark transformation and data-quality pipelines for schema validation, de-duplication, hash-
• based CDC, and reconciliation, ensuring end-to-end traceability and more than 99.3% valid-record acceptance.
• • Optimized Spark and Delta Lake workloads by implementing partition pruning, Adaptive Query Execution
• (AQE), broadcast joins, and Z-Ordering, reducing query run-times and cutting Azure compute costs.
• • Implemented enterprise governance through Unity Catalog & Azure Key Vault, enabling centralized metadata
• management, data lineage, secure credential handling and fine-grained access control across Lakehouse assets.
• • Delivered curated sales, store-performance, and transaction-level KPI datasets modeled for Power BI, enabling
• cross-regional business teams to monitor performance and drive data-backed operational strategies.
• Manufacturing Data Lakehouse & Analytics Platform
• • Engineered and deployed an Azure Databricks Lakehouse platform for a large-scale manufacturing
• organization, ingesting 500K+ incremental ERP records daily from Azure SQL Database utilizing watermarking
• and JDBC predicate pushdown.
• • Built robust Medallion architecture using Databricks and PySpark, delivering business-ready analytical data
• models, aggregate tables, and Databricks SQL endpoints, improving visibility into regional manufacturing trends.
• • Standardized CI/CD practices using Azure DevOps to automate pipeline deployments, maintaining a 99.5%
• pipeline execution success rate and significantly reducing manual deployment effort.
• BFSI Data Migration & ETL Development
• • Orchestrated the complex migration of enterprise applications and transactional data across heterogeneous
• database environments for a leading international bank.
• • Designed strict source-to-target validation and automated reconciliation controls to guarantee zero data loss and
• accurate cutover during critical deployment windows.
• • Optimized legacy Informatica PowerCenter ETL workflows, improving batch processing efficiency, production
• reliability, and maintainability across legacy-to-target data movement.