Azure Data Engineer
VENTAS SOFTECH Pvt Ltd. · Jan 2023 – Present
- Designed and developed end-to-end Azure Data Factory pipelines for batch and incremental data ingestion from heterogeneous retail source systems including POS, ERP, CRM, and e-commerce platforms.
- Built and optimized PySpark notebooks in Azure Databricks to perform large-scale data transformation, cleansing, deduplication, and enrichment on retail transactional datasets.
- Monitored ADF pipeline health using Azure Monitor, configured alerting rules for SLA breach detection, and resolved pipeline failures with root cause analysis and permanent fixes.
- Developed Delta Lake tables with ACID transaction support, SCD Type 2 history tracking, and Z-ordering optimization to support retail analytics and regulatory compliance requirements.
- Implemented Microsoft Fabric Lakehouse architecture with medallion layers (Bronze, Silver, Gold) to organize and govern retail data assets from raw ingestion to analytics-ready serving.
- Maintained comprehensive pipeline documentation, data dictionaries, and runbooks ensuring operational continuity and enabling knowledge transfer across data engineering team members.
- Wrote complex SQL queries, stored procedures, and analytical views in Fabric Warehouse and Azure SQL Database to compute retail KPIs for merchandise, supply chain, and finance reporting.

