Lead Software Engineer - Innovate the Future of AI & ML Data Infrastructure Are you ready to take part in an exciting journey to build a groundbreaking data platform for AI and Machine Learning? Join us in creating a next-generation infrastructure utilizing the open-source data engine, Daft, and be part of a dynamic team.
You will collaborate with top-tier researchers from leading AI laboratories and play a pivotal role in shaping the future of data technology.
Your Role - Develop essential features from user-friendly APIs and dashboards to scalable distributed schedulers and Rust-based execution engines.
- Seamlessly integrate Daft with cutting-edge technologies such as Ray, Apache Iceberg, Unity Catalog, and Delta Lake.
- Embrace a feedback-driven approach to rapidly iterate alongside elite AI researchers and laboratories.
- Engage directly with infrastructure, prototyping, and a user-centered design process.
What You Bring - Over 10 years of engineering experience with a strong foundation in full-stack development, distributed systems, and ML infrastructure.
- A successful history of building innovative products from inception to completion, ideally within startup environments or entrepreneurial initiatives.
- Active participation in open-source projects or personal initiatives showcasing your creativity and technical skills.
- Proficiency in tools like Pandas, NumPy, Scikit-Learn, PyTorch, Spark, and familiarity with systems languages such as Rust or C .
- Ability to translate user insights into practical development strategies.
- A degree in Computer Science from a prestigious institution (e.g., Stanford, MIT, Berkeley, CMU, Caltech, UIUC).
Key Technologies - Languages: Python, Rust, C (Rust/C experience is beneficial but not mandatory)
- Frameworks: Daft, Spark, Ray, Apache Iceberg, Delta Lake
- Cloud: AWS, S3, Unity Catalog