Big Data Engineer

Job Title: Big Data Engineer

Job Location: Irving, TX OR New Castle, DE (Hybrid)

Duration: 12 + Months,

Only banking domain

Long term projects

Job Description:

• 3+ years Working experiences with Starburst/TRINO, Impala, Hive, SparkSQL.

• Good knowledge with hadoop environment and distribution system.

• Strong data analysis, data evaluation and problem-solving skills

• Proficiency in SQL and debugging is crucial.

• Experiences with writing Python, Pyspark

• Identify and troubleshoot data issues and provide timely resolution.

• Great communication skills to explain technical to non tech team member.

• Great skills to understand business requirement and convert into tech requirement with action steps.

• Domain knowledge of data quality/data entitlement is a plus.

• Agile experience.

Responsibilities:

This role is responsible for the delivery of Data Quality rules which includes:

• Code & development & enhance SQL Queries.

• Release the SQL to UAT testing and Prod; Job scheduling and running.

• Production support - job monitoring and SQL error debugging.

• Ad hoc python script creation.

• Hosting meetings and present tech demos.

• Discovery of new technical challenges that can be solved with existing and emerging Big Data hardware and software solutions.

• Assisting and support analyst team on technical solutions.

• Understands the SQLs syntax/functions across different analytic engine/platform, responsible to convert Starburst/impala SQL to SparkSQL by referring spark documents and provide optimal query.

• Works with team members to ensure efforts within owned tracks of work will meet their needs.

• Communication and documentation.