
Software Developer
IBM · Dec 2024 – Present
- • Build and maintain distributed Spark platform services for IBM watsonx.data and Cloud Pak for Data, working across backend, cloud infrastructure, and platform integrations.
- • Led migration of Spark workloads from IBM Analytics Engine (IAE) to watsonx.data Spark, enabling engine-based APIs for job submission, lifecycle management, and platform standardization.
- • Implemented delegated RBAC workflows allowing non-admin users to provision Spark engines by dynamically granting Lakehouse access using Zen service tokens and AMS authorization.
- • Developed webhook-based callback mechanisms for Spark job lifecycle events, enabling real-time application status updates for external orchestration and monitoring systems.
- • Added Amazon S3 MRAP (Multi-Region Access Point) support for Spark, enabling secure multi-region data access using DAS signing without static IAM credentials.
- • Contribute to Integration and Validation Tests, production debugging, and platform reliability, delivering fixes, performance improvements, and security enhancements across Spark services.



