
Data Engineer
Cognizant · Aug 2024 – Present
- Designed and developed robust and scalable big data pipelines using AWS Glue, Bash, hive, Oracle, Apache PySpark.
- Implemented ETL processes to transform and load data from various sources into an Oracle database
- Automated data workflows with Linux scripting, ensuring efficient data processing and error handling, reducing manual effort by 40 percent.
- Worked on Spark and Query Performance Tuning resulting in reducing time taken by 20 percent and AWS Cost optimization
- Processed and analyzed 10TB of batch data using Hadoop, PySpark and Apache Hive for an ATT Payment Analytic engine, resulting in a 20 percent increase in customer engagement
- Utilized PyApache Spark to perform complex transformations and aggregations on 3TB of financial transaction data
- Implemented data quality checks and governance frameworks to ensure accurate and consistent data using bash and Apache Hive scripts
- Led a team of Data Engineers, providing technical guidance, mentoring, training, upskill new team members and architectural direction to deliver high-quality, scalable data solutions aligned with business goals.



