Senior Data Engineer

Job Details:

Title: Sr. Data Engineer

Salary: $180-$230K

Requirements: 8+ years of software dev exp (Large Data Sets or Big Data)

Location: 100% onsite (San Francisco Bay Area)

Our client, a gaming startup, is seeking a Sr. Data Engineer to join their passionate team. The ideal candidate is an experienced engineer who will play a vital role in reconstructing and refining data processes, collaborating closely with the engineering team to guarantee smooth integration of data solutions.

Responsibilities

Comprehend and optimize existing data processes within Databricks.

Collaborate with the engineering team to seamlessly integrate data solutions.

Translate basic product requirements into robust back-end systems.

Demonstrate proactive implementation of solutions to meet project deadlines.

Work on systems involving various stakeholders, requiring effective communication skills.

Provide technical leadership and guide the team on coding best practices, data management, and system architecture.

Requirements

Bachelor's or Master's degree in Computer Science, Software Engineering, or related field.

Senior Back-End Engineer experience with focus on data-related projects.

Proficiency in various data storage solutions including basic RDBMS, key-value stores, data warehouses, and data lakes.

Familiarity with caching systems.

Extensive experience in at least one queue system such as Kafka, RabbitMQ, or similar (understanding of enterprise solutions like Confluent or MSK is a plus).

Basic understanding of open-source technologies in data pipelines, such as Airbyte or Meltano.

Exposure to enterprise solutions like Fivetran or Databricks is advantageous.

Real-time data processing and streaming techniques using Spark structured streaming and Kafka.

Ability to work independently, architect solutions, and perform data ingestion development and support.

Segregation capability between BI analytics and data engineering workflows.