Job Description
Job description
Job Role: Data Engineer
Location: New York City, NY
Duration- 12 Months
Job Description
• Bachelor’s Degree or Master’s in Computer Science, Engineering, Software Engineering or a relevant field.
• 10+ years of software development experience building large scale distributed data processing systems/application or large scale internet systems.
• Must have Good communication skills, writing clean code.
• Strong experience in Python, Kafka, Spark, SQL ,Hive, Couchbase, Impala, Hadoop, Splunk, Autosys, shell scripting etc.
• Experience in DevOps Activity ( Snow, Build (TeamCity, Jenkins) and UDeploy.
• Good knowledge on Capital Market and Data Analysis.
• Experience in dealing with Huge Volume of Data operations.
• Design best architecture and select the most appropriate modeling techniques and data visualization for big data analysis.
• Performance analysis, troubleshooting and resolution (this includes familiarity and investigation of Cloudera/Hadoop logs)
• Experience with Linux-based infrastructures, Linux/Unix administration, and Cloud.
Must To Have
• Strong experience in Python, Kafka, Spark, SQL ,Hive, Impala, Hadoop, Splunk, Autosys, shell scripting etc.
• Data Analysis and Data Wrangling skills when dealing with Huge Volume(1B+ Transaction)
• Performance analysis, troubleshooting and resolution including Cloudera Big Data(PySprak ) eco system.
• Experience with Linux-based infrastructures.
Nice To Have
• Bachelor’s Degree or Master’s in Computer Science, Engineering, Software Engineering or a relevant field.
• Understanding of capital markets and financial derivatives (rates or other)
• Experience in UI design, JavaScript and Angular charting, Graph.
• Experience with CI/CD pipelines
• Experience with Machine learning and Artificial Intelligence is a plus
• Test-driven development, including Unit and End-to-End Testing.
Competencies
• Strong verbal and written communication skills; ability to face off to business users
• Self-motivated individual and with determination to achieve goals
• Willingness to learn, both technically and professionally
• Strong analytical and problem solving skills
• Good team working skills and ability to work in a distributed global team environment
• Ability to work on a fast-pace environment; Flexible and able to deliver quality results in the required timeframe
