sai poorna

+•• ••••• •••••••••••@•••••.com

Azure Data Engineer · Signovate Technologies · 4.8 yrs

As an Azure Data Engineer, I approach my work with determination and meticulousness, driven by a logical and pragmatic mindset. I am dedicated to achieving mastery in my field, diligently crafting robust and efficient data solutions. My structured approach ensures high-quality outcomes, reflecting my commitment to precision and excellence.

  • Open to Bangalore
  • On-site

Ask about sai

Skills

Claimed

Accurate Data DeliveryEnd-to-End Solution OwnershipTechnical DiscussionTechnical Knowledge SharingTeamworkIndependent WorkIncident ManagementTeam SupportTechnical TroubleshootingProcess ImprovementProduction Deployment VerificationCron Job Scheduling

Proof of Experience

Signovate Technologies

Azure Data Engineer

Signovate Technologies · Dec 2024 – Present

  • Proficient Azure Data Engineer with 5+ years of experience in designing, implementing, and optimizing data solutions on Microsoft Azure.
  • Expertise in developing and managing data pipelines, integrating data from multiple sources, and ensuring high-performance, secure, and scalable cloud data architectures
  • Hands-on experience with Azure services such as Azure Data Factory, Azure Storage, Logic Apps, and Azure Databricks, implementing scalable data solutions.
  • Proficient in developing and implementing data lake solutions using Azure Data Factory, enabling efficient data integration and transformation workflows.
  • Extensive experience working with Azure Data Lake Gen2, ensuring secure and optimized data storage for large-scale data operations.
  • Hands-on experience in creating and configuring Azure Self-hosted Integration Runtime to connect on-premises data sources with Azure cloud services.
  • Solid hands-on experience writing Spark SQL queries for data processing, aggregation, and analysis in a distributed environment.
  • Expertise in designing and developing data flows in Delta Lake, enhancing the performance and reliability of data operations.
  • Proficient in data preparation and processing using SQL and Spark SQL, streamlining data workflows for data engineers and analysts.
  • Experienced in creating clusters in Databricks to support large-scale data processing and analytics tasks.
  • Skilled in developing and scheduling jobs in Databricks for automating data pipelines and ensuring seamless data flow.
  • Strong expertise in creating Technical Design Documents, debugging code, and performing unit testing to ensure high-quality deliverables.
  • Extensive experience in developing stored procedures, functions, exception handling, views, dynamic SQL, and complex queries using SQL Server and T-SQL.
  • Good understanding of customer needs and industry trends to deliver tailored solutions.
  • The objective of this project is to reduce the costs and risks involved in preparing reports from data coming from legacy systems.
  • The solution implements a data warehouse to improve reporting timeliness and reduce the gap between business and customers.
  • Understanding the client's business, requirement gathering, and creating a design document.
  • Coordinate with data mapping, data cleansing and DV teams to fulfill requirements.
  • Process data from source to Azure Synapse Analytics as tables.
  • Develop Azure Data Factory pipelines using Copy Activity.
  • Achieve business transformation using Azure Data Factory/PySpark Databricks notebooks using Delta tables.
  • Automate data validation and quality checks using Python and Pandas.
  • Develop PySpark scripts to move data from Data Lake to Azure SQL DB.
  • Create Databricks notebooks for data transformation using PySpark and schedule those jobs.
  • Use Microsoft Azure services including Azure Data Factory, Databricks, Functions, Key Vault, Data Lake and Azure Logic Apps.
  • Use Generative AI (GenAI) to assist with SQL/PySpark development, data analysis, troubleshooting pipeline/code issues, and preparation of technical documentation.
  • Conduct unit testing on pipeline functionality and code base and participate in internal UTA testing.
  • Prepare documents for production move and attend daily client calls for query clarification
  • Lincoln Financial Distributors data warehouse providing wholesalers and sales management access to accurate sales-compensation data and faster turnaround for timely reports through Territory Visualizer reporting.
  • Understanding the client's business, requirement gathering, and creating a design specification document.
  • Create mapping sheets based on requirements provided by the client.
  • Ingest data from Blob Storage (Excel files) to SAP Foundation layer Delta tables using ADF and trigger pipelines.
  • Ingest data from SAP Foundation Delta layer to trusted Delta layer using incremental loading in ADF and perform transformation logic in the trusted layer.
  • Ingest data from trusted Delta layer to unified Delta layer (dimension and fact tables) using SCD Type 1 logic in ADB and perform unit testing/data validation.
  • Closely work with Power BI developers on data-related issues.
  • Use cloud connectors including Azure Synapse (SQL DW), Snowflake, AWS, Azure Data Lake Store, Azure Blob Storage, Oracle and SQL Server.
  • Navistar telematics monitoring solution supporting batch processing, near real-time IoT data analysis and Power BI dashboards for device health and maintenance monitoring.
  • Azure Data Factory processed batch data from six different cloud and on-premises sources every two hours.
  • Data wrangling on near real-time IoT data for Azure Synapse for data analysis.
  • Power BI dashboards with 20+ reports covering device health and maintenance monitoring.
  • Involved as Technical Consultant and translated functional design into technical design.
  • Understanding the client's business, requirement gathering, and creating design documents.
  • Develop Azure Data Factory pipelines to move data from on-premises databases to Azure Data Lake Storage and Azure SQL Database.
  • Develop Azure Data Factory pipelines using Copy Activity and Stored Procedure Activity.
  • Created solutions using Databricks Spark/HDInsight batch to extract and load data on Azure Data Lake Store and used on-premises data gateway to load data into Azure Analysis Service.
  • Achieve business transformation using Azure Data Factory/PySpark Databricks notebooks using Delta tables.
  • Develop PySpark scripts to move data from Data Lake to Azure SQL DB.
  • Create Databricks notebooks for transformation using PySpark and schedule those jobs.
  • Use Microsoft Azure Cloud with Azure Data Factory, Databricks, Functions, Key Vault, Data Lake, Azure Logic Apps and CI/CD.
  • Conduct unit testing on pipeline functionality and code base.
  • Automate deployment using Azure DevOps CI/CD methodology.

Azure Data Engineer

Bosch · Nov 2023 – Nov 2024

Data Engineer

BSW SOFT · Apr 2021 – Feb 2023

Proof-of-Work

No completed sessions yet. Anything sai finishes shows up here with the evidence behind it.

Proof of Education

B.Tech, ECE

JNTU Anantapur University

A Proof CV — one profile, kept current, with the proof attached. Back to the portal