Date Posted: 06/07/2024
Hiring Organization: Rose International
Position Number: 465521
Job Title: Data Engineer, Engineering Data Management
Job Location: San Francisco, CA, USA, 94115
Work Model: Hybrid
Employment Type: Temporary
Estimated Duration (In months): 7
Min Hourly Rate ($): 90.00
Max Hourly Rate ($): 117.24
Must Have Skills/Attributes: AWS, Py Spark, Python, SQL
Job Description
Only those lawfully authorized to work in the designated country associated with the position will be considered.
Please note that all Position start dates and duration are estimates and may be reduced or lengthened based upon a client's business needs and requirements.
Only qualified Data Engineer candidates located in the San Francisco, CA area will be considered due to the position requiring an onsite presence
Education Requirement:
• A Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
Minimum Qualifications:
• 9+ years of experience in a Data Engineering role
• 5+ years of hands-on experience in Snowflake
• Mandatory 5+ years of working in dbt with knowledge on advanced dbt concepts like macros and Jinja templating
• Minimum 4+ years of work experience in B2B Marketing Domain
• Advanced working SQL experience working with relational databases, query authoring (SQL) and working familiarity with a variety of databases
• Experience with scripting languages such as Python
• Experience with big data tools such as PySpark
• Experience with AWS cloud services used often for data engineering including S3, EC2, Glue, Lambda, RDS, or Redshift
• Experience working with APIs to pull and push data
• Experience optimizing big data data pipelines, architectures and data sets
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
• Able to read existing code to understand functionality and debug issues
Preferred Qualifications:
• Familiarity with Agile and SCRUM methodologies is a plus
• Experience working with PowerBI to develop dashboards is a plus
• Analytical skills related to working with unstructured datasets
• A successful history of processing value from large, disconnected datasets
• Experience working with agile, globally distributed teams
As a Lead Data Engineer, you will build the foundations that enable data driven marketing and the digital transformation journey at Client. You will work with data strategists to understand the business requirements and execute the tactical aspects of delivering on the vision. Your team will drive multiple tool selection initiatives, ingesting data from different source systems to the data lake, Data Wrangling to enable downstream data consumers, developing systems & processes to improve data reliability, developing a comprehensive data catalog and training downstream data consumers. You would also be responsible for leading the deliverables of your juniors as well as taking important decisions relevant to platform architecture and you will report to Sr. Manager, Software Development.
Data Engineer Responsibilities:
• Create and maintain optimal data pipeline architecture
• Assemble complex data sets that meet functional / non-functional requirements
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, Dbt and AWS big data technologies
• Build analytics tools that utilize the data pipeline to provide actionable insights into employee experience, operational efficiency, and other key business performance metrics
• Work with stakeholders to assist with data-related technical issues and support associated data infrastructure needs
• Build processes supporting data transformation, data structures, metadata, dependency and workload management
• Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS and Azure) and find ways to apply them back help their team
• Work with data scientists and analysts to strive for greater functionality in our data systems
• Lead a team of data engineers and provide technical guidance and mentorship
Possibility of extension and/or contract to hire.
Benefits:
For information and details on employment benefits offered with this position, please visit here. Should you have any questions/concerns, please contact our HR Department via our secure website.
California Pay Equity:
For information and details on pay equity laws in California, please visit the State of California Department of Industrial Relations' website here.