Azure Data Engineer
Hexaware · Apr 2026 – Present
- Designed and developed scalable ETL pipelines using Azure Databricks and PySpark to process healthcare claims, member, and provider data.
- Implemented Medallion Architecture (Bronze, Silver, Gold) with Delta Lake and Unity Catalog to enable secure, governed data access.
- Developed reusable metadata-driven Databricks notebooks for ingestion, transformation, validation, and incremental data loading.
- Optimized Spark jobs through partitioning, caching, and Delta optimization techniques, reducing pipeline execution time.
- Worked closely with business analysts and reporting teams to deliver trusted datasets for operational and analytical reporting.
- Performed data profiling, cleansing, standardization, deduplication, and business-rule validation across healthcare claims and related datasets.
- Integrated and harmonized multiple healthcare datasets into curated, analysis-ready data layers for analytics and reporting.
- Used SQL and PySpark transformations to support business data requests and analytical dataset preparation.


