Are you excited about innovating in the tech industry? At Capital One, we're looking for passionate individuals who thrive on solving complex business challenges in a dynamic and collaborative environment. Join us as a Lead Data Engineer and take part in transforming our data landscape.
As a key member of our product data team, you'll be responsible for creating and maintaining advanced data pipelines that process information from major auto clients. Leveraging cutting-edge AWS services such as EMR, S3, and Databricks, your work will enhance the operations of thousands of auto dealerships across the U.S. and unlock the potential of data-driven insights.
What You'll Do: - Contribute to designing and building a scalable, low-latency, fault-tolerant streaming data platform.
- Develop the next generation of distributed streaming data pipelines and analytics data stores using frameworks such as Flink and Spark Streaming with programming languages like Python, Java, or Scala.
- Collaborate with engineers to construct data pipelines utilizing big data technologies (Spark, Flink, Kafka) on medium to large-scale datasets.
- Engage in an agile work environment that emphasizes continuous integration/deployment, resiliency standards, and collaboration with cyber & security teams.
- Stay ahead of tech trends and explore new technologies while also mentoring fellow engineers.
- Work closely with Agile teams to develop, test, implement, and support technical solutions across the full stack.
- Partner with experts in machine learning, microservices, and full-stack systems to drive innovation.
- Collaborate with product managers to deliver robust cloud-based solutions that enhance digital experiences for customers seeking financial empowerment.
- Perform unit tests and peer reviews to ensure your code is robust, efficient, and high-quality.
Basic Qualifications: - Bachelor's Degree
- At least 3 years of experience in application development (internship experience does not apply)
- At least 1 year of experience with big data technologies
Preferred Qualifications: - 5+ years of application development experience in Python, SQL, Scala, or Java
- 2+ years of public cloud experience (AWS, Azure, Google Cloud)
- 3+ years with distributed data computing tools (Kafka, Spark, Flink)
- 2+ years working on real-time data and streaming applications
- 2+ years of NoSQL implementation experience (DynamoDB, OpenSearch)
- 2+ years of data warehousing experience (Redshift or Snowflake)
- 3+ years of experience with UNIX/Linux, including basic commands and shell scripting
- 2+ years of experience with Agile engineering practices
Location: This position is based in Plano, TX, with a salary range of $144,200 - $164,600 for Senior Data Engineer.
Please note that Capital One is unable to sponsor new employment authorization applications for this role.
Capital One values diversity and is an equal opportunity employer, committed to creating an inclusive environment for all applicants. We encourage you to apply if you meet the qualifications.