Senior Data Engineer

Job Description

A software engineer who will be involved with the design, and responsible for development, implementation and on-going support of .NET applications/python as well as database ETL processes. The position will be comprised work closely with data science. Should have experience with .Net development of web services and using Git version control repositories. Experience with dependency injection, unit testing, and software design patterns.

Should have knowledge of and experience with Machine Learning and AI concepts and be able to articulate differences between them and data requirements that they may have. Particularly the candidate should have knowledge of NLP based services, such as Chat GPT/OpenAI, and experience of prompt engineering, embeddings, fine tuning etc. via the api. Should understand the flow of data and be able to extract/clean data, into appropriate structures, in order to train models. Should understand synchronous and asynchronous data pipelines and training methodologies.

Must be able to work on multiple projects simultaneously, including both enhancements as well as new project development. The software engineer's responsibilities will be to design and develop these applications, and to coordinate with the rest of the team working on different layers of the infrastructure. The candidate must be a self-starter with a sense of urgency and a commitment to quality and professionalism.

Responsibilities:

Work with Technical Lead to ensure implementation follows best practices

Review and refine stories and use cases into functional applications

Design, build, and maintain efficient, reusable, and reliable C code

Integrate with AI models, 1st and 3rd party

Integrate with 1st and 3rd party API's for data consumption

Integrate with data storage solutions such as SQL Server/Spark/Vector DB/Search

Ensure the best possible performance, quality, and responsiveness of applications

Identify bottlenecks and bugs, and devise solutions to mitigate and address these issues

Integration of the front-end and back-end aspects of the web application

Collaborate with other team members and stakeholders