Sr. Big Data Engineer

Role: - Sr. Big Data Engineer
Location : - Jersey City, NJ (Onsite)
Duration : - Long term Contract
Position: - Full-time
INTERVIEW MODE : 2 rounds - VIDEO Technical Interview with CODING + HM Interview.
MUST SKILLS : Spark, Scala, Python

Technical & Other Skills

• 7+ years of working experience in scalable data pipeline using Scala, Python, pyspark, Hadoop, Apache Spark, Spark SQL, Kafka, Nifi, ETL and Incremental Data Load with Big data technologies.

• Experience of working with Databases like Oracle, Netezza and have strong SQL knowledge

• Developing scalable streaming solutions based on Hadoop/ Big Data stack such as HDFS, Ozone, Hive/ Impala, Apache Spark and Kafka.

• Experience in Data Engineering and implementing multiple end-to-end DW projects in Big Data environment

• Experience of building data pipelines through Git, Shell, Spark with Java/Python on Hadoop or Object storage (S3/CEPH)

• Strong analytical skills required for debugging production issues, providing root cause and implementing mitigation plan

• Strong communication skills - both verbal and written - and strong relationship, collaboration skills and organizational skills

• Flexible enough to stretch and work in India time to coordinate with India team

Experiences

• Nice to have development experience in Python based AI/ML workloads on Hadoop.

• Experience of working on Nifi, APIs and Unix Environment will be an added advantage

• Experience of working in Agile teams

• Ability to multi-task across multiple projects, interface with external / internal resources and provide technical leadership to junior team members

• Ability to be high-energy, detail-oriented, proactive and able to function under pressure in an independent environment along with a high degree of initiative and self-motivation to drive results

• Ability to quickly learn and implement new technologies, and perform POC to explore best solution for the problem statement

• Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance standards and improve process efficiency

• Perform assigned tasks and production incident independently