Big Data Hadoop Engineer

Title: Big Data Hadoop Engineer
Location: Pleasanton, CA
Duration: 12 months+
Compensation: $52/hr - $74/hr
Work Requirements: , Holders or Authorized to Work in the U.S.

ABOUT THE ROLE:
The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long-term IT strategy and planning to include high-level analysis, professional reports and presentations, and mentoring, support and training.

CURRENT SITUATION:
We are looking for consultant who can provide vision and technical leadership on the project. The resource will review the existing the architecture and recommend new technologies as appropriate. Hadoop engineer will closely work with business units and recommend changes to provide value to business.

TASKS:

1. Provide vision, gather requirements and translate client user requirements into technical architecture.
2. Design and implement an integrated Big Data platform and analytics solution
3. Design and implement data collectors to collect and transport data to the Big Data Platform.
4. Implement monitoring solution(s) for the Big Data platform to monitor health on the infrastructure

RESOURCE REQUIREMENTS, SKILLS, KNOWLEDGE AND ABILITIES:
Resource assigned to the project have the minimum skills requirement to render the services in a competent and efficient manner.

Technical Knowledge and Skills:

• Strong project experience in Big Data, Cloudera Distribution 7.x, Cloud migration, RDBMS is must.
• Strong project experience with Amazon EMR/Databricks/Cloudera CDP is must.
• 4-5 years of programming experience in Python, Java and Scala is must.
• Strong experience with REST API development using Python frameworks ( Django, Flask etc.)
• Micro Services/Web service development experience using Spring framework
• Strong experience in Amazon EMR/DataBricks/Cloudera CDP
• Experience with Dask, Numpy, Pandas, Scikit-Learn
• Hands-on experience in Big Data, Cloudera Distribution 7.x, RDBMS
• Hands-on experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL., HBase.
• Strong experience working in Real-Time analytics like Spark/ Kafka/ Storm
• Experience with Jenkins, JIRA
• Expertise in Unix/ Linux environment in writing scripts and schedule/execute jobs.

PROFESSIONAL SKILLS:

• Strong analytical skills with the ability to analyze information and identify and formulate solutions to problems.
• Provide more in-depth analysis with a high-level view of goals and end deliverables.
• Complete work within a reasonable time frame under the supervision of a manager or team lead.
• Plan and manage all aspects of the support function.
• Extensive knowledge of and proven experience with data processing systems, and methods of developing, testing and moving solutions to implementation.
• Strong knowledge in project management practices and ability to document processes and procedures as needed.
• Work collaboratively with other support team members and independently on assigned tasks and deliverables with minimum supervision
• Communicate effectively with users at all levels, from data entry technicians up to senior management, verbally and in writing.
• Self-motivated, working closely and actively communicating with team members to accomplish time critical tasks and deliverables
• Ask questions and share information gained with other support team members, recording and documenting this knowledge
• Elicit and gather user requirements and/or problem description information, and record this information accurately
• Listen carefully and act upon user requirements
• Convey and explain complex problems and solutions in an understandable language to both technical and non-technical persons
• Present technical solutions to management and decision makers
• Follow the lead of others on assigned projects as well as take the lead when deemed appropriate
• Think creatively and critically, analyzing complex problems, weighing multiple solutions, and carefully selecting solutions appropriate to the business needs, project scope, and available resources
• Take responsibility for the integrity of the solution.

Must Haves-

• Strong project experience in Big Data, Cloudera Distribution 7.x, Cloud migration, RDBMS is must.
• Strong project experience with Amazon EMR/Databricks/Cloudera CDP is must.
• 4-5 experience building data pipelines using Hadoop components Sqoop, Hive, Solr, MR, Impala, Spark, Spark SQL., HBase.
• 4-5 years of programming experience in Python, Java and Scala is must.
• Strong experience with REST API development using Python frameworks (Django, Flask etc.)
• Micro Services/Web service development experience using Spring framework is highly desirable

CORE COMPETENCIES:

• Act with integrity
• Use sound judgement
• Commitment to quality
• Demonstrate adaptability
• Innovate
• Think strategically
• Communicate effectively and influence others
• Work well both independently and as part of a team

Our benefits package includes:

• Comprehensive medical benefits
• Competitive pay
• 401(k) retirement plan
• and much more.

About INSPYR Solutions

Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at

INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities.