Machine Learning

We are seeking a talented Machine Learning/ETL Engineer to join our IT team and take on the challenge of advancing our Business Intelligence initiatives. In this role, you will be responsible for designing, developing and optimizing ETL/ELT processes for large-scale data processing and leveraging Machine Learning frameworks to perform data quality and anomaly detection across 1st and 3rd party datasets. You will work closely with analysts, data scientists, and other stakeholders to understand business requirements and translate them into high quality solutions. As a key member of our Business Intelligence IT team, you will leverage your expertise in machine learning algorithms, data engineering, and data analytics to drive ETL/ELT solutioning from a wide variety of datasets. You will play a critical role in maintaining data integrity and ensuring the timely availability of data for analytical purposes. We are looking for someone who is not only technically proficient but also possesses strong problem-solving skills and a passion for continuous learning in the rapidly evolving business.

Responsibilities

1. Architecting and implementing robust, high-performance data pipelines across various data sources (databases, APIs, web services) to efficiently process and transform large datasets.
2. Enable and ensure data collection from various sources and loading it to a data lake, data warehouse, or any other form of storage.
3. Proficiently utilizing cloud platforms Snowflake, AWS, GCP to build scalable data processing solutions, including managing data lakes and data warehouses.
4. Developing and implementing data quality checks and monitoring systems to ensure data accuracy and reliability throughout the data pipeline.
5. Contribute to data documentation and governance strategies, including data access controls, data lineage tracking, and data retention policies.
6. Collaborating closely with data analysts, data scientists, and business stakeholders to understand data needs, translate requirements into technical solutions, and deliver actionable insights.
7. Excellent analytical and problem-solving skills, with the ability to work independently and in a team environment.

Qualifications

1. 7+ years of experience with data warehouse technical architectures, ETL/ELT, reporting/analytic tools, and scripting.
2. Extensive knowledge and understanding of data modeling, schema design, and data lakes.
3. 5+ years of data modeling experience and proficiency in writing advanced SQL and query performance tuning with on Snowflake in addition to Oracle, and Columnar Databases SQL optimization experience.
4. Experience with AWS services including S3, Lambda, Data-pipeline, and other data technologies.
5. Experience implementing Machine Learning algorithms for data quality, anomaly detection and continuous monitoring, etc.

Required Skills and Experience

1. Strong proficiency in Python, SQL for data manipulation and processing.
2. Experience with data warehouse solutions for Snowflake, BigQuery, Databricks.
3. Ability to design and implement efficient data models for data lakes and warehouses.
4. Familiarity with CI/CD pipelines and automation tools to streamline data engineering workflows.
5. Deep understanding of principles in data warehousing and cloud architecture for building very efficient and scalable data systems.
6. Experience with Apache Airflow and/or AWS MWAA.
7. Experience with Snowflake's distinctive features, including multi-cluster architecture and shareable data features.
8. Expertise in distributed processing frameworks like Apache Spark, or other big data technologies is a plus.

Requirements

1. Bachelor's degree in Computer Science, Information Technology, or a related field.
2. Proven experience as an ETL Engineer or similar role in data engineering.
3. Strong understanding of ETL concepts, tools, and best practices.
4. Proficiency in programming languages such as SQL, Python.
5. Experience with ETL tools/platforms Databricks, IBM DataStage is a major plus.
6. Knowledge of data warehousing concepts and architecture.
7. Ability to work collaboratively in a team and manage multiple tasks effectively.

Salary Range: $90,000-$140,000 a year