MD Danish

MD Danish

As a Lead Engineer, I am driven by a strong ambition to tackle complex challenges, applying a meticulous and logical approach to every project. My diligent work ethic and resourcefulness ensure I consistently deliver high-quality, effective solutions, always striving for precision and optimal outcomes.

Work style

hybrid, full-time

Identity verified by DigiLocker
Contact details — on requestProof of Work — on request

Experience

C

LEAD ENGINEER-I

CitiusTech Healthcare Private Limited · Apr 2025 – Present

Not yet confirmed
  • Designed and developed metadata-driven ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks, PySpark, and Azure Synapse Analytics for scalable healthcare data integration.
  • Engineered complex transformation logic using SQL Stored Procedures, aligning business rules to deliver high-quality data to target tables and file-based outputs.
  • Developed SQL Stored Procedures, Functions, and transformation logic to process data across Stage, Ingress, and Egress layers.
  • Built and optimized PySpark and Spark SQL notebooks for data ingestion, transformation, validation, and automated generation of client-specific Egress files.
  • Automated vendor-specific file generation processes using SQL Stored Procedures, supporting multiple client facilities and improving operational efficiency and consistency.
  • Engineered Ingress frameworks to ingest data from SFTP/ADLS, perform historical backfill loads, automate file validation, and ensure reliable end-to-end data processing.
  • Orchestrated end-to-end data workflows using ADF, integrating Databricks notebooks, SQL procedures, and multi-source data pipelines into Azure Synapse Analytics.
  • Used Azure DevOps to manage Agile/Kanban boards, track PBIs, maintain project documentation, and coordinate production support activities.
  • Managed production ETL operations by monitoring pipeline health, resolving job failures, performing root cause analysis, implementing corrective actions, and ensuring reliable end-to-end data processing.
  • Supported Microsoft Fabric migration by assessing ETL workflows, validating migration readiness, documenting data assets, and assisting in migration planning and testing.
  • Partnered with business and cross-functional teams to deliver client enhancements, implement change requests, prepare technical documentation, and support seamless production deployments.
CitiusTech

LEAD ENGINEER-I

CitiusTech · Apr 2025 – Present

Not yet confirmed
  • • Designed and developed end-to-end Azure data pipelines using Azure Data Factory (ADF), Azure Databricks, SQL Server, and Azure Data Lake Storage (ADLS) to automate ingestion, transformation, and delivery of structured and semi-structured data.
  • • Built metadata-driven ETL frameworks to process Excel, CSV, and TXT files, leveraging SQL Server stored procedures and Databricks notebooks for scalable data transformation and file generation.
  • • Implemented incremental and full-load data ingestion strategies, enabling efficient processing of source files from ADLS into SQL Server target tables while maintaining data consistency and performance.
  • • Developed reusable and dynamic SQL Server stored procedures for automated table creation, schema-driven data processing, and file generation, reducing manual effort from 7–8 hours to under 30 minutes.
  • • Engineered Databricks-based data processing solutions for bulk file ingestion, transformation, validation, and export, improving throughput and operational efficiency across multiple data feeds.
  • • Created automated data quality monitoring and notification frameworks using Databricks notebooks, proactively alerting stakeholders about unmapped fields.
  • • Orchestrated complex ETL workflows through Azure Data Factory, incorporating dependency management, error handling, parameterization, scheduling, and end-to-end process automation.
  • • Performed ad-hoc data analysis, data validation, and root-cause investigations on large datasets, collaborating with business stakeholders and clients to resolve data discrepancies and support decision-making.
V

Data Engineer

Veritas Comunications Private Limited · Aug 2021 – Feb 2025

Not yet confirmed
  • Designed and developed ELT pipelines using Azure Databricks, Azure Data Factory (ADF), and Azure Synapse Analytics to process structured data.
  • Built scalable ETL pipelines using Azure SQL Database, ADF, and PySpark for data ingestion, transformation, and loading into analytical data stores.
  • Implemented Delta Lake and Delta Tables to provide ACID-compliant storage, schema enforcement, and reliable data processing.
  • Developed and optimized PySpark and Spark SQL transformations for large-scale data processing and improved query performance.
  • Designed end-to-end data solutions using Azure Data Lake Storage (ADLS Gen2) with Azure Databricks to support batch and streaming workloads.
  • Developed data models and transformation logic using SQL and Python to support reporting, analytics, and business intelligence requirements.
VERITAS COMUNICATIONS PRIVATE LIMITED

Data Engineer

VERITAS COMUNICATIONS PRIVATE LIMITED · Aug 2021 – Feb 2025

Not yet confirmed

Skills 0 proven through work

Also works with

AI Identity VerificationProcess RerunningTransaction Log ManagementDeletion Vectors UsageRescued Data Column UsageMetadata-Driven ETLRequirements AnalysisAnalytical ThinkingClient FocusData-Driven RecommendationsStrategic PlanningSoftware Architecture PrinciplesConfiguration-Driven System DesignModel Training ExecutionAI Agent DevelopmentDashboard UI DesignArtificial Intelligence FundamentalsProcess ImprovementChatbot DevelopmentRetrieval-Augmented GenerationData Mart DevelopmentGo ProgrammingLarge Language Model ConceptsStakeholder ManagementSystem EnhancementBug FixingMetadata-Driven Feature InterpretationApplication Performance OptimizationFault-Tolerant System DesignConcurrency OptimizationData ExportJSON Data HandlingSchema Drift ManagementACID PropertiesDelta Live Tables UsageProblem SolvingTechnical TroubleshootingError HandlingDefining ConstraintsData Integrity ManagementData ValidationPHI RedactionHashing AlgorithmsDate FormattingAzure Synapse Analytics UsageAzure Data Lake UsagePySparkQuery WritingData LoadingData CurationStar Schema DesignStored Procedure DevelopmentData TransformationDatabase ManagementMedallion Architecture DesignRaw Data Layer DesignSFTP File Transfer ManagementData EngineeringRevenue Cycle OperationsDatabricks UsageAzure Data Factory Pipeline DevelopmentETL Pipeline Development

Proof of Work

Proof of Work

MD shares this with people who ask. You'll hear back either way.

Education

B.E., Civil Engineering

JAIN UNIVERSITY, BANGALORE, KA, IND · 2010 — 2014

Contact details

Contact details

MD shares this with people who ask. You'll hear back either way.

Ask about this candidate

AI responses may contain errors. Proof status and source information are provided by Proof of Skill.

A Proof CV — one profile, kept current, with the proof attached. Back to the portal