Data Engineer

Job profile : Data Engineer experience on Spark with Scala / Azure Databricks

Job Location : DEERFIELD, IL

Job Description :

Competencies:

Minimum 4-6 years of experience in Data Warehousing and Analytics

" Minimum 3-4 years of experience on Spark with Scala / Azure Databricks

" Hands on experience of building data pipelines using Apache Spark (Azure Databricks), Scala

" Hands on experience working on Databricks batch.

" Hands on experience in Azure Data Factory.

" Hands on experience in designing and delivering solutions using the Azure Data Analytics platform including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake,

" Experience working on Delta Lakes.

" Experience working in a Dev/Ops environment with tools such as Azure DevOps

" Should be able to work independently on assigned projects with offshore team.

Desirable Skills:

" Knowledge of Talend Tool.

" Exposure to Azure PaaS services is highly desirable.

" Problem-solving and good analytical skills.

" Communication, teamwork and negotiation skills.

Work in Agile development projects for batch and real time data ingestion, transformation.

Work closely with product owners, understand requirements, lead offshore team on Development and QE of data pipelines.

Work with upstream/downstream teams to resolve any issues/blockers.

Work with System Integration/Performance engineering team for integration/performance testing of data pipelinesCoordinate with DevOps/RunOps teams for deployment in higher environments

Regards

Shubham Chandra

Sr. US IT Recruiter

Diverse Lynx LLC 300 Alexander Park Suite Princeton , NJ 08540

Office: (phone number removed) Ext330

Email: URL:

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.