Business Development LLP
Summary
Skilled and experienced Sr. Data Engineer with proven experience in Spark, Kafka, Hadoop, Data Management.
Experience
- Minimum 7 years of experience.
- B.Sc. of Computer Science or similar.
Must Have Skills/Technologies (Mandatory)
- Proficiency in programming languages Scala, Java, Python.
- Experience with design and development of ELT workflows and data pipelines for high-volume data systems e.g. Datastage, Informatica, Talend, etc.
- Experience in Data Engineering and implementation of Real-time data processing pipelines using Kafka, Kafka Streams and Spark Streams.
- Good analytical skills with excellent knowledge of SQL, data management and relational databases like PostgreSQL, Oracle, MySQL.
- Experience working with Big Data Tools/Technologies and DataLakes (Hadoop, Map/Reduce, Spark, Hive).
- Knowledge/Experience working with In-memory/NoSQL databases - Redis, HBase, Cassandra, CouchBase, CouchDB.
- Knowledge/experience with Apache Nifi/Airflow for Data Pipelines Orchestrations.
- Proven expertise with Linux administration and Unix scripting for the automation of tasks, Containerization (Kubernetes, Docker, Openshift).
- Experience of building CI/CD tools (Jenkins).
- Experience with version control systems like Git.
Plus Skills/Technologies (Nice to have)
- AI and Machine Learning knowledge, Database Administration, Big Data Storage, Data Analytics and Feature Engineering, ELK Stack, FLink.
- Familiar with cloud platforms like AWS, Google Cloud, or Microsoft Azure.
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Engineering and Information Technology
Industries
IT Services and IT Consulting