Job Description
We are seeking a skilled Data Engineer responsible for developing and delivering data solutions to accomplish technology and business goals and initiatives. Key responsibilities include performing code design and delivery tasks associated with the integration, cleaning, transformation, and control of data in operational and analytical data systems. Job expectations include working with stakeholders and Product and Software Engineering teams to aid with implementing data requirements, analyzing performance, and researching and troubleshooting data problems within system engineering domains.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: .
Skills and Requirements
Experience with Kafka, Batch, Elasticsearch
Strong SQL Skills one or more of MySQL, HIVE, Impala, SPARK SQL
Data ingestion experience from message queue, file share, REST API, relational database, etc. and experience with data formats like json, csv, xml
Experience working with SPARK Structured steaming
Experience working with Hadoop/Big Data and Distributed Systems
Experience working with admin/prod support/debugging
Working experience with Spark, Sqoop, Kafka, MapReduce, NoSQL Database like HBase, SOLR, CDP or HDP, Cloudera or Hortonworks, Elastic Search, Kibana, etc.
Hands on programming experience in at least one of Scala, Python, PHP, or Shell Scripting, to name a few
Performance tuning experience with spark /MapReduce and/or SQL jobs
Experience and proficiency with Linux operating system is a must
Experience in end-to-end design and build process of Near-Real Time and Batch Data Pipelines
Experience working in Agile development process and deep understanding of various phases of the Software Development Life Cycle
Experience using Source Code and Version Control systems like SVN, Git, Bit Bucket etc.
Experience working with Jenkins and Jar management
Self-starter who works with minimal supervision and the ability to work in a team of diverse skill sets
Ability to comprehend customer requests and provide the correct solution
Strong analytical mind to help take on complicated problems
Desire to resolve issues and dive into potential issues
Ability to adapt and continue to learn new technologies is important Experience with Elasticsearch
Experience working with SPARK Structured steaming
Performance tuning experience with spark /MapReduce or SQL jobs
Experience in end-to-end design and build process of Near-Real Time and Batch Data Pipelines
Experience working in Agile development process and deep understanding of various phases of the Software Development Life Cycle
Experience using Source Code and Version Control systems like SVN, Git, Bit Bucket etc.
Experience working with Jenkins and Jar management null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to .
Date Posted: 19 March 2025
Apply for this Job