Harshal Mane

+•• ••••• •••••••••••@•••••.com

Senior Data Engineer · Startappss System India PVT LTD · 5.4 yrs

As a Senior Data Engineer, I am driven by a deep ambition to achieve mastery in crafting reliable and impactful data solutions. I take full accountability for my work, ensuring high-quality outcomes that reflect my idealistic vision for technology's positive influence, while valuing the autonomy to innovate and deliver with precision and empathy.

  • Remote
  • In 15 days

Ask about Harshal

Skills

Claimed

Security AutomationWorkflow Design (Process Mapping)API Authentication ImplementationRoot Cause AnalysisCloud Architecture AdvisoryCode ReviewDatabase Query OptimizationApplication Performance OptimizationTechnical Solution DesignTechnical TroubleshootingRisk and Controls AssessmentDependency Analysis

Proof of Experience

Senior Data Engineer

Startappss System India PVT LTD · Jul 2021 – Present

  • Architected and implemented a scalable Medallion Architecture (Bronze-Silver-Gold) on AWS for WTW's insurance data platform, processing 45+ GB of daily batch and streaming data across policy, claims, and risk datasets.
  • Designed hybrid batch + streaming ingestion pipelines from on-prem SQL Server, RDS, and Kafka (MSK) into S3 data lake, using AWS Lambda for format conversion and enabling near real-time data availability.
  • Built large-scale transformation jobs using AWS Glue (PySpark) in the Silver layer for data cleansing, SCD Type 2 historical tracking, and incremental processing, significantly improving data reliability and performance.
  • Developed reusable Terraform modules for multi-region infrastructure, S3 replication, and IAM governance, and delivered analytics-ready Gold layer aggregates with Athena views for downstream reporting and API consumption.
  • Architected and deployed a fully automated credential rotation solution for Snowflake using AWS Lambda, Secrets Manager, and Terraform, eliminating 100% manual secret management and strengthening security compliance across production environments.
  • Implemented a secure multi-stage rotation workflow (create, set, test, finish) with RSA/JWT authentication and cross-account ECR policies using awsSourceArn conditions for controlled CI/CD access.
  • Configured multi-region ECR replication and Terraform-driven infrastructure for Lambda, IAM roles, and scheduled rotations, ensuring high availability and seamless deployments.
  • Designed and orchestrated modular Airflow DAGs to ingest change data streams from multiple source databases into Snowflake, enabling near real-time data availability for analytics teams.
  • Built robust end-to-end ETL pipelines (raw
  • staging
  • reporting) with data validation, bad-record handling, and MERGE-based upsert logic, improving data accuracy and reducing manual reconciliation efforts.
  • Implemented data quality checks, snapshot tracking, and historical retention mechanisms, while exposing key pipeline metrics for observability and faster troubleshooting.

Proof-of-Work

No completed sessions yet. Anything Harshal finishes shows up here with the evidence behind it.

Proof of Education

B.Tech, Information Technology

Chameli Devi Group of Institutions, Indore · 2019 — 2023

A Proof CV — one profile, kept current, with the proof attached. Back to the portal