Job Title: - Azure Data Engineer
Location: - Remote
Duration: - 6+ Months
Experience: - 11+ years
Primary duties:
- Support overall Azure Cloud Ecosystem at Client through management of data stores, data pipelines and analytics output
- Manage Operational Azure Synapse environments and Data Lakes
- Manage and Develop Integrations (Data Factory pipelines, APIs, FTP)
- Manage Event and Messaging Queues (Event Hubs)
- Build out advanced analytics use cases from sourcing data through building analytics and piping and delivering data outputs to applications and visualization toolsets
- Partner with Infrastructure, Support and Application teams to identify and implement best practices for cloud services monitoring and management, and promoting these best practices across Spire
- Research and implement cloud services to support cloud apps and maintain cloud services
- Support POC Cloud Application Development
- Monitor cloud app performance for potential bottlenecks and resolving performance issues
- Partner with Infrastructure in the identification and implementation of cost-saving strategies to reduce ongoing cloud expenses, with specific responsibility in cloud data services costs
- Automate key services and tasks across cloud systems to increase efficiency and further reduce cloud costs
- Assist in the formulation of recovery plans and support the execution of the plan
- Work with Information Security to support the security model for data on cloud
- Perform detailed analysis, design and development of the logical and physical data models
- Engage with business teams to help architect and develop use cases and data models used in analytics and visualizations tools and other applications
- Work with Enterprise Architecture teams to develop data governance and data quality layers within the Azure platform
- Works in conjunction with the Advanced Analytics, BI and Infrastructure (Networking, Azure Support, Security) teams
- All other duties as assigned
Qualifications:
- Ability to demonstrate the core and job category competencies: customer-focused, results-driven, teamwork, communications, adaptability, initiative, problem solving and quality
- Ability to lead by example in support of the Company's essential characteristics and values: strong ethics and integrity, dedication to a safe work environment, commitment to a diverse workforce that recognizes and respects all individual's unique skills and perspectives
- Ability to work independently and manage time effectively in order to meet individual goals and deadlines.
Supervisory responsibilities:
Education and experience:
- Bachelor's degree and / or approximately 5+ years of related work experience; or a graduate degree and approximately 3+ years of related work experience
- At least 3 years of experience in developing data ingestion, data processing and analytical pipelines for big data, relational databases, data warehouse and analytics solutions
- Extensive hands-on experience implementing data migration and data processing using Azure services: Azure Datalake, Azure Data Factory, Synapse/DW, Azure SQL DB, Event Hub, IOT Hub, Azure Stream Analytics, Azure Analysis Service, Data Catalog, Client Studio, AI/Client, etc.
- Well versed in DevSecOps and CI/CD deployments within an Azure environment
- Experience developing and deploying ETL pipeline solutions on Azure
- Familiarity with the technology stack available in the industry for metadata management: Data Governance, Data Quality, MDM, Lineage, Data Catalog etc.
- Experience working with Developer tools such as Azure DevOps, GitLabs, Jenkins, etc.
Experience with SQL, ADLS/Blob, Python etc.