Pavan Polagani

Hyderabad

As a Data Engineer with 8 years of experience, I approach complex challenges with diligence and a pragmatic, logical mindset, always driven by curiosity to find the most effective solutions. I am ambitious in building robust data architectures and thrive in collaborative environments where open-mindedness leads to innovative outcomes.

Experience

H

Data Engineer

Helson Software Solutions Private Limited · Sep 2024 – Apr 2026

Not yet confirmed

• Designed and implemented automated, HIPAA-compliant data ingestion and transformation pipelines using Azure Data Factory, Azure Databricks, and PySpark to manage end-to-end claim adjudication data flows, reducing data latency by 35%.
• Ingested and parsed structured and semi-structured health insurance data, including HIPAA EDI 837, 835, and 834 files, processing over 15M daily records into a Medallion Architecture (Bronze to Silver to Gold).
• Mapped legacy healthcare data onto modern HL7 FHIR data profiles within Databricks to standardize provider and patient records, establishing a unified data layer for downstream Predictive Machine Learning models.
• Built modular DBT models to clean, anonymize, and execute dynamic data masking on Protected Health Information (PHI), improving data compliance audit speeds by 40% while standardizing raw medical and pharmacy claims.
• Leveraged DBT to architect optimized Star and Snowflake schemas for Member Eligibility Dimensions, Provider Networks, and Institutional/Professional Claim Fact tables.
• Aligned processed health data layers with CMS guidelines to support Hierarchical Condition Category (HCC) coding, actuarial risk adjustment factor (RAF) scores, and capitation payment modeling, supplying clean feature sets for AI-driven healthcare risk forecasting.
• Developed transformation logic to extract and compute clinical data metrics required for HEDIS quality scorecards, improving query execution times by 25% for downstream business intelligence and analytics tracking.
• Implemented rigorous DBT data validation tests for membership enrolment overlaps, duplicate claims detection, and cross-reference validation to guarantee 100% financial and billing accuracy before consumption.

Data Engineer

HELSON SOFTWARE SOLUTIONS PRIVATE LIMITED · Sep 2024 – Apr 2026

Not yet confirmed

Associate Product Developer

EdgeVerve · Aug 2022 – Sep 2024

Not yet confirmed

E

Associate Product Developer

Edgeverve Systems Limited · Aug 2022 – Sep 2024

Not yet confirmed

• Led the end-to-end data migration strategy across AWS and Azure to consolidate over 4 TB of core banking records, KYC compliance data, and multi-tenant portfolios into a centralized Snowflake environment.
• Orchestrated automated ELT/ETL pipelines via Azure Data Factory to securely ingest raw customer verification documents, US CDD attributes, and checking/savings account master tables into ADLS Gen 2.
• Built scalable PySpark and Spark SQL workloads in Azure Databricks to process global payment streams, utilizing Delta Lake formats to process global payment streams including US ACH, European SEPA, and ISO 20022 messages, scaling the system to handle 1,200 transactions per second.
• Engineered high-performance data warehouse layers across Snowflake, Azure Synapse, and Amazon Redshift, configuring multi-cluster virtual warehouses to reduce compute costs by 18% under heavy, concurrent analytical workloads.
• Leveraged DBT to architect optimized Star and Snowflake schemas for KYC Status, Account Type, and Loan Portfolio Dimensions alongside unified Transaction Fact tables.
• Implemented Snowflake Dynamic Data Masking and row-level security to protect sensitive PII and financial records, aligning data access with US SARs, European GDPR/PSD2, and AML regulations, while preparing text data for secure Large Language Model (LLM) processing and RAG frameworks.
• Automated data quality frameworks using DBT tests and balance cross-checks to generate clean, validated datasets for downstream AI-driven credit risk modeling and automated fraud compliance monitoring.

T

Analyst

Techsoft Solutions Private Limited · Aug 2021 – Aug 2022

Not yet confirmed

• Collaborated with stakeholders and retail analysts to translate complex business transformation requirements into scalable technical data solutions on AWS.
• Designed, developed, and maintained automated data pipelines using AWS Glue and PySpark to manage massive volumes of daily point-of-sale (POS) transactions, handling an average of 50 GB of new data per day.
• Spearheaded data ingestion and validation workflows, implementing strict quality checks to ensure reliable outputs for product catalogue and inventory monitoring.
• Optimized data transformations and Snowflake queries across varying sales data volumes by applying wide/narrow Spark transformations, parallel processing, and partitioning, improving query execution efficiency by 30%.
• Processed multi-source retail data in CSV and Parquet formats on AWS S3, standardizing on Parquet to reduce downstream cloud storage footprint by 22%.
• Configured and orchestrated AWS Glue jobs for end-to-end processing from S3 ingestion to Snowflake loading to support complex retail analytics queries.
• Built modular DBT models in Snowflake (staging, intermediate, marts) to transform raw sales records into analytics-ready product performance data marts, creating highly optimized pipelines used for AI-powered demand forecasting and recommendation engines.

Analyst

TECHSOFT SOLUTIONS PRIVATE LIMITED · Aug 2021 – Aug 2022

Not yet confirmed

Analyst/Software Engineer

Capgemini · Apr 2021 – Aug 2021

Not yet confirmed

• Mastered enterprise-level training on Azure cloud infrastructure, data platform services, and big data engineering architectures.
• Developed foundational proficiency in PySpark and advanced SQL, building and optimizing complex analytical queries to manipulate large-scale datasets.
• Designed and tested basic, end-to-end ETL/ELT data pipelines, focusing on automated data ingestion, transformation logic, and loading workflows into target data environments.

Analyst

Capgemini · Apr 2021 – Aug 2021

Not yet confirmed