Role: Senior Azure Data Engineer/Data Engineer III
Location: Jackson, MI (100% remote)
Duration: Long term
Rate: Market
Job Description: If you're looking for a career that will provide a challenging work environment and many opportunities to learn, our company is the place for you. Join our team and become part of a company full of exciting opportunities. We help employees achieve their personal best by offering many opportunities that develop and enhance their skills for career advancement.
As a Senior Azure Data Engineer, your role involves designing, implementing, monitoring, and maintaining data and analytical solutions on Microsoft Azure.
Data Pipeline Design and Implementation: - Create and maintain data pipelines that facilitate data movement, transformation, and integration.
- Ensure efficient and reliable data flow from source systems to target destinations.
Data Storage Solutions: - Design and manage data storage solutions within Azure, including:
- Azure Data Lake: Storing large volumes of raw data.
- Azure SQL Database: Providing structured data storage.
- Azure Blob Storage: Handling unstructured data.
Data Processing and Transformation: - Expertise in Databricks and dbt to process and transform data.
- Leverage Azure Databricks for running dbt transformations in production.
Data Integration: - Collaborate with data analysts and scientists to understand data requirements.
- Design effective data workflows that enable data-driven decision-making.
Performance Optimization: - Optimize SQL queries generated by dbt for better performance.
- Ensure consistency between development (using a Databricks SQL warehouse) and production (using Azure Databricks compute) environments.
Security and Compliance: - Implement secure and compliant data processing pipelines.
- Use Azure services and frameworks to produce cleansed and enhanced datasets for analysis.
Skills: - Expertise in Microsoft Azure, Azure Databricks, and DBT
- Knowledge of data catalog tools (e.g., Microsoft Purview)
- Proficiency in data processing languages, including SQL and Python
- Excellent problem-solving skills and ability to work in a collaborative environment.
Education: - Bachelor's degree in computer science, Engineering, or related field.
- Certified Azure Data Engineer
- Experience as an Azure Data Engineer or similar role.
Skills and Experience: Required Skills: - Microsoft Azure
- Data pipelines
- Data integration
- Python
- Performance optimization
Additional Skills: - SQL
- Datasets
- Maintain data
- Blob
- SQL queries
Problem-solving