Job title: Senior Data Engineer
Job Responsibilities: - Design and develop the pipelines required for optimal extraction, real-time decisioning, reporting, optimization, data collection, transformation, and loading data from a wide variety of data sources using SQL, Python and Azure services.
- Apply modification techniques to transform raw data into compatible formats for downstream systems. Utilize software and computing tools to ensure data quality and completeness. Implement code to extract and validate raw data from upstream sources, ensuring accuracy and reliability.
- Collaborate with stakeholders to document and understand data requirements. Evaluate project plans to assess data costs, access, and availability. Draft design specifications to model data flow and storage, ensuring data is easy to connect and manage.
- Adhere to data modeling and handling procedures to maintain compliance with laws and policies. Document data type, classifications, and lineage to ensure traceability and govern data accessibility.
- Perform root cause analysis to identify and resolve anomalies. Implement performance monitoring protocols and build visualizations to monitor data quality and pipeline health. Support and monitor data platforms to ensure optimal performance and compliance with service level agreements.
- Implement code to extract raw data, validate its quality, and ensure the correct data is ingested within your area of work after cooking, data transformation.
- Follow data modeling and handling procedures to maintain compliance with all applicable laws and policies across your assigned workstreams.
- Develop and implement solutions for data quality validation and continuous improvement.
- Manage, execute, and monitor weekly and monthly production operations; resolve and escalate production issues as appropriate.
- Partner with data scientists, PMs, engineers, and business stakeholders to understand business and technical requirements, plan and execute projects, and communicate status, risks, and issues.
- Develop user-friendly dashboards for various applications, such as National IT Reporting, Spend Analytics, Scorecards, Supply chain analytics, Incident and Service Level Agreement (SLA) Compliance Monitoring, Spares and Inventory Management, and other business-facing applications.
- Manage the delivery of high impact dashboards, data visualizations and manage SLA's for all data sets and processes running in production
Degree Requirement: This position requires, at a minimum, a bachelor's degree in computer science, computer information systems, information technology or a combination of education and experience equating to the U.S. equivalent of a Bachelor's degree in one of the aforementioned subjects.
Work Schedule: Full Time, 40 Hours/Week