Spark Developer RapidBrains

Job Title: Spark Developer

Experience Level: 6 - 10 Years

Location: Bangalore/Hyderabad

Position Type: Full-time

Notice Period - 0 - 15 Days

Employment type: Permanent

Job Summary:

We are seeking a highly skilled Spark Developer to join our dynamic team. The ideal candidate will have extensive experience in developing, deploying, and managing Spark jobs within Kubernetes environments. You should possess a deep understanding of Spark on Kubernetes, as well as strong expertise in Kubernetes and Docker concepts.

Key Responsibilities:
• Develop, deploy, and manage Spark jobs in Kubernetes environments.
• Optimize Spark jobs for performance and scalability in Kubernetes clusters.
• Implement best practices for Spark on Kubernetes for efficient job execution.
• Collaborate with cross-functional teams to integrate Spark jobs with other data processing systems.
• Monitor and troubleshoot Spark job performance issues within Kubernetes clusters.
• Automate the deployment and scaling of Spark jobs using Kubernetes and Docker.
• Maintain and improve existing Spark applications for better efficiency and reliability.
• Document processes, configurations, and best practices for Spark jobs in Kubernetes.

Skills & Requirements:
• Proficient in developing, deploying, and managing Spark jobs in Kubernetes.
• In-depth knowledge of Spark on Kubernetes for job execution.
• Strong understanding of Kubernetes architecture and components.
• Expertise in Docker, including creating and managing Docker containers.
• Familiarity with CI/CD pipelines and tools (e.g., Jenkins, GitLab CI/CD) for automating Spark job deployments