prem kumar

Data Engineer | SQL Expert • PySpark • Azure Data Factory • ADLS Gen2 • Delta Lake | Building Scalable ETL/ELT Pipelines on Azure | Medallion Architecture | · Gulbarga, Karnataka, India

As a Software Engineer, I am driven by a meticulous and logical approach, consistently striving for mastery and accountability in every project. My pragmatic mindset, combined with a strong ambition, fuels my dedication to optimizing complex data solutions and ensuring high-performance outcomes.

Experience

Software Engineer - 2

NetM Corp. · Jun 2025 – Present

Not yet confirmed

• As a Software Engineer II, I design, develop, and maintain cloud-based ETL/ELT solutions on Microsoft Azure.
• My work focuses on building scalable data pipelines using Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Storage Gen2 (ADLS Gen2), Azure SQL Database, PySpark, and Delta Lake to support enterprise reporting and analytics.
• My responsibilities include developing parameterized Azure Data Factory pipelines, orchestrating data ingestion from multiple source systems, implementing PySpark transformation workflows, and building Delta Lake pipelines using the Medallion Architecture (Bronze, Silver, Gold).
• I also develop incremental data loading frameworks using Delta Lake MERGE operations and optimize Spark workloads through partitioning, caching, broadcast joins, OPTIMIZE, and ZORDER techniques.
• Additionally, I design and optimize Azure SQL database objects, perform source-to-target validation and reconciliation, support production data pipelines, troubleshoot pipeline failures, and collaborate with cross-functional teams to deliver reliable and maintainable enterprise data solutions.

Software Engineer

NetM Corp. · Mar 2024 – May 2025

Not yet confirmed

• As a Software Engineer I (Data Engineering), I contributed to the development and maintenance of cloud-based ETL/ELT solutions on Microsoft Azure.
• I worked closely with senior data engineers to build scalable data pipelines, automate data integration workflows, and support enterprise reporting and analytics.
• My responsibilities included developing and maintaining Azure Data Factory (ADF) pipelines for data ingestion, integrating Azure Databricks notebooks for data transformation, and processing data using PySpark and Delta Lake.
• I participated in implementing Medallion Architecture (Bronze, Silver, Gold), building incremental data loading frameworks using Delta Lake MERGE operations, and transforming raw data into analytics-ready datasets.
• I developed reusable PySpark modules for data cleansing, schema validation, aggregations, joins, and business transformations while working with Spark SQL and DataFrame APIs.
• I also created and optimized Azure SQL database objects, including tables, views, stored procedures, and queries to support reporting and downstream applications.
• Additionally, I performed source-to-target data validation, monitored production pipelines, investigated execution failures, resolved data quality issues, and collaborated with cross-functional teams to ensure reliable and consistent data delivery.
• I used Git and GitHub for version control and participated in code reviews and deployment activities following established development practices.
• This role strengthened my expertise in Azure Data Factory, Azure Databricks, ADLS Gen2, PySpark, Delta Lake, Azure SQL Database, ETL/ELT development, and production support while building scalable enterprise data engineering solutions.

Associate Software Engineer

Infogain · Mar 2021 – Feb 2024

Not yet confirmed

• As a Data Engineering Intern / Associate Engineer, I developed Python-based ETL utilities to automate data ingestion, preprocessing, validation, and loading of structured datasets into MySQL databases.
• I worked extensively with Python, Pandas, and SQL to build reusable ETL workflows, implement configurable data validation rules, create SQL queries and stored procedures, optimize database operations, and support ETL processes for reporting and operational systems.
• This role strengthened my foundation in Python programming, SQL development, database design, and data engineering best practices.