Role: Backend & Data Engineer - Search Platform
Location: Remote
Experience: 3+ Years
About the Role
We are looking for a highly skilled Backend & Data Engineer to join our Search Platform team. You will be responsible for building scalable data pipelines, integrating multimedia analysis tools, and implementing search capabilities to power a robust and intelligent information retrieval system.
Key Responsibilities
- Design and optimize data pipelines to extract and transform metadata from PDFs, documents, and other unstructured sources using OCR tools like Textract and Tesseract.
- Integrate image, video, and audio analysis workflows using tools such as OpenCV, AWS Rekognition, and Whisper.
- Index both structured and unstructured data using Elasticsearch or OpenSearch for high-performance search capabilities.
- Build and maintain RESTful APIs to provide client access to indexed data.
- Implement semantic and vector search using tools such as FAISS or Weaviate.
- Ensure tenant-specific data isolation, access control, and search result accuracy.
Skill Set & Qualifications
- 3-6 years of experience in backend development and/or data engineering roles.
- Strong proficiency in Python (preferred), especially for API development and ETL pipeline creation.
- Hands-on experience with search engines like Elasticsearch, OpenSearch, or similar technologies.
- Demonstrated ability to handle and process unstructured data - including text, images, and videos.
- Familiarity with vector-based search techniques and libraries such as FAISS, Weaviate, etc.
- Experience with cloud infrastructure (preferably AWS) and containerized deployments using tools like Docker, ECS, Lambda, etc.
- Excellent collaboration skills and the ability to thrive in a fast-paced, agile environment with tight deadlines.