Azure Data Engineer
Synthima Technologies Pvt Ltd · Aug 2023 – Aug 2026
- Designed and managed end-to-end ETL/ELT pipelines using Azure Data Factory to ingest healthcare data from NHS Digital systems, clinical applications, and external sources.
- Built scalable ADLS Gen2 storage for structured, semi-structured, and unstructured healthcare datasets and Implemented Bronze, Silver, and Gold Medallion Architecture using Azure Databricks and Delta Lake.
- Developed PySpark notebooks for large-scale cleansing, standardization, aggregation, complex joins, validation, and transformation.
- Integrated SQL Server and legacy NHS systems into Azure Synapse Analytics with schema consistency, reconciliation, and query optimization.
- Managed the migration of legacy on-premises SSIS packages to Azure Data Factory, converting package-level Control Flow and Data Flow logic into scalable, reusable ADF pipeline components.
- Re-engineered SSIS-based ETL workloads by mapping Execute SQL, Lookup, Conditional Split, Derived Column, and file-processing logic to ADF activities and parameterized stored procedures.
- Validated SSIS-to-ADF migration results through source-to-target reconciliation, record-count validation, data-quality checks, and parallel-run comparison before production cutover.
- Implemented Auto Loader and DLT pipelines for incremental ingestion, declarative ETL, data-quality expectations, and reliable transformations.
- Developed Microsoft Fabric pipelines for Lakehouse and Data Warehouse environments and integrated Snowflake for enterprise analytics.
- Implemented Azure Event Hubs for real-time ingestion and Unity Catalog for centralized governance, access control, and metadata management.
- Integrated curated data with Power BI and implemented datasets, DAX measures, dashboards, scheduled refreshes, and row-level security.
- Implemented CI/CD using Azure DevOps and supported monitoring, troubleshooting, production releases, and stakeholder collaboration.

