Data Engineering Manager (Architecture Architect Solutions Engineer Java Python Automation Data Lake Datalake Data Mesh CI/CD Big Data AWS SQL Oracle Java Kafka Apache Iceberg Hudi Finance Trading Financial Services Banking Remote Working Governance Management Regulation Hedge Fund Investment Management) required by our financial services client in Manhattan, New York City.
You MUST have the following:
- Good experience as a hands-on Data Engineering Manager/Architect/Technical Lead
- Excellent design and architecture ability for enterprise-scale data solutions
- Java - doesn't have to be recent
- Enterprise-scale SQL or Oracle
- Agile
The following is DESIRABLE, not essential:
- Kafka or equivalent for streaming
- Python
- Amazon Web Services (AWS) or GCP or Azure
- Experience delivery projects in data management, governance and regulation
- Apache Iceberg
- An understanding of data mesh architecture
Role: Data Engineering Manager (Architecture Architect Solutions Engineer Java Python Automation Data Lake Datalake Data Mesh CI/CD Big Data AWS SQL Oracle Java Kafka Apache Iceberg Hudi Finance Trading Financial Services Banking Remote Working Governance Management Regulation Hedge Fund Investment Management) required by our financial services client in Manhattan, New York City. You will be hired to be the technical lead and co-manager of a new team that is being assembled to build a new data management platform on AWS. The greenfield project will include the automation of data catalogue population and the implementation of data governance policies. You will be the lead engineer/manager/solutions architect in a team that has a senior developer, mid-level developer and business lead. You and the business lead will share responsibility of the team. He will be responsible for the interpretation of data regulation, building of road maps and strategy and creation of policies. You will do the design, architecture and technical delivery of this strategy and his data policies. Over the course of the next year, you will hire more developers into the team and the workload grows. You will be working with Java and Python on AWS with Apache Iceberg. As the data environment moves from an AWS based data lake to a data mesh architecture, any understanding of data mesh would also be highly desirable.
You will also contribute to the two other teams in the data engineering space within the company- the data platform team which operates a Hudi based data lake and the team working with Iceberg and Kafka to create the new data mesh architecture- but the data governance programme will be your priority.
Hours are 8.30am - 5.30pm. Hybrid working is 2 days/week in the office.
Salary and Comp: $170k - $220k + 50% Bonus + 401k