Big Data Engineer

Job Title: Big Data Engineer Location: Bangalore Experience: 7+ Years Contract Duration: 12 Months Must have skill: (Hadoop, Python, PySpark) Job Description: We are seeking experienced Big Data Engineers with strong expertise in Hadoop, Python, and PySpark to join our dynamic team for a long-term contract. The ideal candidate should have a solid understanding of distributed computing, data pipeline design, and large-scale data processing. Key Responsibilities: Develop and maintain scalable and reliable big data pipelines using Hadoop, Python, and PySpark Optimize and troubleshoot Spark jobs for performance and efficiency Collaborate with data scientists, analysts, and other stakeholders to understand data requirements Integrate data from various sources, ensuring consistency and quality Implement data governance, security, and privacy best practices Perform unit testing and validation of data pipelines and processes Participate in design reviews and contribute to architecture decisions Technical Skills Required: Strong experience in Hadoop ecosystem (HDFS, Hive, YARN, etc.) Proficient in Python programming for data manipulation and scripting Hands-on experience with PySpark for distributed data processing Experience with data ingestion tools and frameworks (Kafka, Sqoop, Flume, etc.) Good understanding of performance tuning and optimization techniques in Spark Familiarity with version control tools like Git and CI/CD pipelines Preferred Qualifications: Experience with cloud platforms (AWS, Azure, or GCP) for data processing Knowledge of SQL and NoSQL databases Familiarity with workflow orchestration tools (Airflow, Oozie, etc.) Excellent problem-solving and communication skills Education: Bachelor�s or Master�s Degree in Computer Science, IT, or related field Job Type: Contractual / Temporary Contract length: 12 months Schedule: Day shift Work Location: On the road