
Data Engineering Senior Analyst
Accenture · Nov 2023 – Present
- Engineered Azure-based enterprise data platforms using Azure Data Factory, Azure Databricks, Azure Synapse Analytics, ADLS Gen2, PySpark, Spark SQL and Delta Lake, supporting large-scale retail domains including Customer, Stores, Supply Chain, Pricing, Inventory and Master Data.
- Designed and implemented end-to-end ELT pipelines by integrating 12+ heterogeneous data sources (databases, APIs, flat files and enterprise applications) into a centralized Unity Catalog-governed Delta Lake platform, utilizing Time Travel to support data versioning and ensure secure, high-quality datasets for analytics and reporting.
- Optimized a 200+ TB enterprise data platform containing structured and semi-structured data (Delta, Parquet, CSV, JSON) using Spark and Delta Lake techniques including partitioning, caching, broadcast joins, Deletion Vectors, OPTIMIZE, Z-Ordering, Auto Compaction, Schema Evolution, indexing and query tuning, improving pipeline performance by 50% and query performance by 30%.
- Designed scalable enterprise data warehouse solutions using Azure Synapse Analytics, implementing dimensional modeling, incremental loading, Serverless SQL Pools and optimized storage strategies to improve scalability while reducing cloud storage and compute costs.
- Owned and supported 60+ production data pipelines, implementing monitoring, SLA tracking and alerting using Azure Monitor and Log Analytics, ensuring high platform availability, proactive incident detection and faster root cause resolution.
- Led cloud migration initiatives from legacy platforms to Azure, including schema design, historical backfilling, partitioning, data validation and system decommissioning, modernizing enterprise reporting and analytics platforms.
- Delivered optimized PySpark/T-SQL transformations, stored procedures, views, functions, indexing strategies and performance tuning to improve data processing efficiency and support high-volume analytical workloads.
- Managed Git-based source control, branching, pull requests and Azure Data Factory CI/CD processes, while collaborating with the Azure DevOps team to enable automated deployments.
- Collaborated with BI, Data Science, Strategy, Product and Business teams in Agile/Scrum (JIRA/Confluence) to deliver trusted datasets, improve dashboard performance and enable self-service analytics.
- Designed and developed a Databricks-based Agentic AI application using RAG to help developers troubleshoot pipeline failures, perform RCA and lineage tracing across pipelines/code/logs - reducing debugging time by 70%.
- Architected a hybrid AI solution integrating Databricks Vector Search with live SQL execution, automating enterprise knowledge ingestion, indexing and semantic retrieval while enabling non-technical users to query enterprise data and engineering knowledge through natural language.
- Led a team of 5 Data Engineers, providing technical guidance, code reviews, production support and delivery planning across multiple data platform initiatives.








