Lead Data Engineer

iSphere is actively seeking a dynamic Lead Data Engineer in Houston for a flex opportunity, specializing in the Power & Gas Commodity markets. This hands-on role involves developing full-stack applications to facilitate Front Office Trading activities, emphasizing the creation of enterprise-wide solutions. As a tech enthusiast, you will collaborate with end-users to analyze requirements, design innovative solutions, and deliver substantial business value.

Key Responsibilities:

- Develop and maintain the Front Office Data warehouse on Amazon Web Services (AWS).

- Design and implement scalable data pipelines to meet organizational data needs.

- Lead a team of data engineers in creating, testing, and deploying data pipelines and ETL processes.

- Collaborate with various teams to understand data requirements and provide effective solutions.

- Establish and enforce best practices for data engineering processes.

- Evaluate and implement new technologies to enhance data pipeline efficiency.

- Monitor and optimize data system performance for reliability, scalability, and cost-effectiveness.

- Provide technical leadership and mentorship to junior team members.

- Troubleshoot and resolve issues related to data pipeline failures and performance bottlenecks.

- Work closely with DevOps and infrastructure teams to manage data systems in production environments.

- Stay informed about industry trends and incorporate emerging technologies to improve existing systems.

Skills and Experience:

- Bachelor's or Master's degree in Computer Science or equivalent experience.

- 10+ years of experience in designing, building, and deploying large-scale data pipelines and ETL processes.

- Background in Power & Gas Commodities is advantageous.

- Proficient in programming languages like Python.

- Expertise in SQL Server, Database Administration, data modeling, and database design.

- Familiarity with modern data warehouse platforms such as Snowflake and Data Bricks.

- Experience with AWS cloud platform.

- Knowledge of containerization technologies like Docker and orchestration tools such as Kubernetes.

- Previous leadership or mentoring experience is preferred.

- Relevant certifications like AWS Certification are a bonus.

If you are a seasoned Lead Data Engineer with a passion for technology and a track record of delivering impactful solutions, we encourage you to apply and join our innovative team at iSphere.