SK

Sharath Kumar

As a data engineer with three years of experience, I am driven by a logical and pragmatic approach to problem-solving, diligently pursuing mastery in my field. I am accountable for delivering robust, high-impact data solutions, always striving for optimal system performance and continuous improvement.

Contact details — on requestProof of Work — on request

Experience

S

Data Engineer

Sai Technologies Pvt Ltd · Apr 2023 – Present

Not yet confirmed
  • Designed and deployed DLT pipelines to automate ETL workflows, enabling real-time data ingestion, transformation, and improved data quality.
  • Automated DLT deployments through CI/CD pipelines using Azure DevOps and Databricks Repos to enforce version control and promote changes across environments.
  • Utilized Databricks Autoloader to efficiently ingest large-scale cloud data into Delta tables, supporting incremental loads and reducing latency.
  • Implemented CDC frameworks using Delta Lake to track source-system changes, ensuring timely updates and consistent data across analytics environments.
  • Optimized Delta tables through strategic partitioning, Z-Ordering, and vacuuming, boosting query performance and lowering storage costs.
  • Built scalable ETL pipelines using ADF, Azure Databricks, and Python to cleanse, transform, and load data into Azure Data Lake and Azure Synapse Analytics.
  • Developed real-time ingestion pipelines with Azure Stream Analytics and Event Hubs to process IoT, sensor, and web-log data for near-real-time insights.
  • Automated data integration workflows via ADF scheduling and Azure Logic Apps, reducing manual intervention and operational overhead.
  • Integrated data from Azure Blob Storage, SQL Server into Azure Data Lake Storage to support unified processing and analytics.
  • Managed large-scale data storage solutions in Azure Data Lake and Azure SQL Database, optimizing, partitioning, indexing and compression for faster queries.
  • Designed star schema models to support BI workloads, ensuring optimal performance for complex analytical queries.
  • Led large-scale data processing using Apache Spark in Databricks, leveraging PySpark for transformations and aggregations, improving processing speed by 30%.
  • Implemented Delta Lake features including ACID transactions and schema evolution to enhance data reliability and governance.
  • Monitored and tuned pipeline performance using ADF Monitoring and Databricks metrics to ensure high-availability and efficient execution.
  • Collaborated with data scientists, analysts, and architects to define data requirements and deliver insights that enabled data-driven decision-making.
  • Partnered with DevOps teams to implement CI/CD pipelines in Azure DevOps, automating deployment and integration of data workflows.
  • Utilized Azure Monitor and Log Analytics to track pipeline health, detect anomalies, and proactively address performance issues.
  • Enforced data governance policies with Unity Catalog in streaming environments, incorporating data quality checks, schema validation, and error-handling controls.
  • Defined data retention and archiving policies in Azure Data Lake Analytics to ensure compliant lifecycle management of real-time data streams.
  • Built production-grade data transformations and stored results in managed Unity Catalog tables.
I

Data Engineer

iPlace India Pvt Ltd · Jul 2021 – Mar 2023

Not yet confirmed
  • Designed and automated end-to-end ETL pipelines in Azure Data Factory (ADF) to extract and process clinical trial data from Veeva Vault and other external sources.
  • Ingested raw clinical datasets into Azure Data Lake Storage (ADLS) and built reusable, parameterized ADF pipelines with automated scheduling, logging, and fault-tolerant execution.
  • Implemented secure data workflows using Azure Key Vault, RBAC, and compliance controls for handling HIPAA-regulated medical data.
  • Delivered consolidated datasets to analytics and BI teams, enabling improved insights into patient progress, trial metrics, and site performance.
  • Performed clinical data validation, cleansing, and quality checks to support FDA regulatory submissions and data accuracy standards.
  • Created ADF linked services, datasets, and dataflows based on business and technical requirements.
  • Built and managed activity dependencies, triggers, and control flows within ADF pipelines.
  • Migrated data from on-premises databases to Azure SQL Database using ADF integration runtime.
  • Developed and executed multi-source data migration strategies using ADF for structured and semi-structured datasets.
  • Scheduled, monitored, and optimized data pipelines for reliable data movement from source to destination systems.
  • Performed transformations using ADF Mapping Data Flows and wrangling transformations for data enrichment and preparation.
  • Troubleshot data ingestion, validation, and transformation issues across ADF and Azure SQL environments.
  • Automated movement of CSV files from Azure Blob Storage to Azure SQL Server using ADF pipelines.
  • Installed and configured Self-Hosted Integration Runtime (IR) with high availability for secure on-premise to cloud connectivity.
  • Managed Databricks code deployments across branches and supported CI/CD processes.
  • Executed delta loads and incremental ingestion strategies during migration using ADF.
  • Set up alerts and monitoring for ADF pipelines to track failures, performance, and SLA compliance.
E

Data Analyst

Exceed Management Pvt Ltd · Mar 2020 – Jan 2021

Not yet confirmed
  • Developed and maintained scalable data pipelines to extract data from multiple sources, perform transformations, and load it into IP Fact using Azure Data Factory.
  • Configured data ingestion processes to synchronize the data from various sources (Flat Files, Databases, REST APIs, etc.) into the Admissions integration platform using Cloud Sync.
  • Built data pipelines utilizing activities, data flows, and transformations to extract and standardize raw data based on business logic.
  • Implemented an end-to-end In-patient (IP) Admissions solution for Health Care messaging system.
  • Developed a robust data processing methods and ETL/ELT workflows using Azure Data Factory (ADF) to construct the IP datasets.
  • Handled data processing involving various source file formats such as CSV.
  • Supported in designing, integrating, and deploying solutions.
  • Proficiency in monitoring and troubleshooting various Azure jobs and data issues to improve the performance and cost efficiency.
  • Created a job scheduler with trigger-based automation for data processing tasks, boosting operational efficiency.
  • Set up email notifications to deliver timely alerts and updates, ensuring dependable task execution and streamlined workflows.
  • Managed CI/CD pipelines and branches to migrate code across environments.

Skills 0 proven through work

Also works with

Data QuarantiningSchema Drift ManagementData Freshness ManagementProfessional Skill DevelopmentTechnical Skill DevelopmentReceiving and Incorporating FeedbackIssue EscalationTransparent CommunicationOn-time DeliverySoftware Quality AssuranceReliabilityCode ReviewDesign ReviewTechnical Knowledge SharingApplying Best PracticesCode OptimizationClean Code PracticesMaintainability DesignDependency AnalysisRequirements AnalysisAdaptabilityWork Planning and PrioritizationScalable System DesignIndependent WorkTaking OwnershipTeamworkProblem SolvingData MonitoringData Architecture DesignCloud ComputingExploratory Data Analysis (EDA)Technical DocumentationCross-Departmental CoordinationData Integrity ManagementIncident ManagementStakeholder Communication AdaptationAnalytical Report CreationMonitoring AutomationApplication Error LoggingData CatalogingSchema ComparisonData ValidationDashboard UI DesignData CurationBusiness Rules VerificationData Unification and NormalizationNotification System DevelopmentError HandlingTask SchedulingData PreprocessingData Repetition PreventionFile Format SupportObject Storage ManagementDatabase Query OptimizationData RecoveryTechnical TroubleshootingDataset VersioningData GovernanceData ModelingACID PropertiesTabular Data Cleaning & ValidationData Partitioning and Segregation DesignData LoadingMedallion Architecture DesignLarge Dataset HandlingData TransformationData CollectionETL Pipeline DevelopmentSpark SQLPySparkDatabricks UsageAzure Data Factory Pipeline Development

Proof of Work

Proof of Work

Sharath shares this with people who ask. You'll hear back either way.

Education

Bachelor's Degree

Kakatiya University

Contact details

Contact details

Sharath shares this with people who ask. You'll hear back either way.

Ask about this candidate

ClaudeChatGPTPerplexity

AI responses may contain errors. Proof status and source information are provided by Proof of Skill.

A Proof CV — one profile, kept current, with the proof attached. Back to the portal