We are seeking highly energetic and collaborative Senior Data Engineers with good exposure to GCP for a 12-month engagement in Sunnyvale (onsite).
Must have skills - GCP
- scala
- spark
- data modelling
- Python
Responsibilities - Design and develop big data applications using the latest open-source technologies.
- Work in an offshore model and manage outcomes effectively.
- Develop logical and physical data models for big data platforms.
- Automate workflows using Apache Airflow.
- Create data pipelines using Apache Hive, Apache Spark, and Apache Kafka.
- Provide ongoing maintenance and enhancements to existing systems and participate in rotational on-call support.
- Quickly learn our business domain and technology infrastructure, and actively share your knowledge with others on the team.
- Mentor junior engineers on the team.
- Lead daily standups and design reviews.
- Groom and prioritize backlog using JIRA.
- Act as the point of contact for your assigned business domain.
Requirements - 4 years of recent GCP experience.
- Experience building data pipelines in GCP.
- Proficiency with GCP Dataproc, GCS, and BigQuery.
- 12 years of hands-on experience developing data warehouse solutions and data products.
- 6 years of hands-on experience with distributed data processing platforms like Hadoop, Hive, or Spark, and workflow orchestration solutions like Airflow.
- 5 years of hands-on experience in modeling and designing schema for data lakes or RDBMS platforms.
- Experience with programming languages such as Python, Java, and Scala.
- Experience with scripting languages like Perl and Shell.
- Experience working with, processing, and managing large data sets (multi TB/PB scale).
- Exposure to test-driven development and automated testing frameworks.
- Background in Scrum/Agile development methodologies.
- Ability to deliver on multiple competing priorities with minimal supervision.
- Excellent verbal and written communication skills.
- Bachelor's Degree in computer science or equivalent experience.
The most successful candidates will also have experience in the following:
- Gitflow
- Atlassian products - BitBucket, JIRA, Confluence, etc.
- Continuous Integration tools such as Bamboo, Jenkins, or TFS.