Job Summary We are looking for experienced Python SQL Developer with expertise in Python, SQL, and Azure cloud technologies to support a critical data warehouse migration project. The selected candidates will work on new data pipelines, data migration, transformations, query conversion, and validation tools in Azure. This is an exciting opportunity to work on cutting-edge technologies and contribute to the migration of data and systems from GCP to Azure.
Key Responsibilities - Data Pipeline Development: Develop and optimize new data pipelines in Azure using Azure Data Factory, Azure Synapse, Event Hub, and AKS.
- Data Migration & Transformation: Lead data migration efforts and implement necessary data transformations to ensure smooth transition from GCP to Azure.
- Query Conversion & Optimization: Convert and optimize queries for the Azure environment.
- Data Validation Tools: Build and implement new data validation tools to ensure data integrity post-migration.
- API Integration: Develop and integrate data APIs in Azure AKS to support the new data platform.
- Performance Tuning: Conduct performance tuning to ensure optimal data processing performance in the new environment.
- Post-MVP Support: Provide documentation and support after the MVP (Minimum Viable Product) release, including additional scope and optimization.
Required Qualifications Experience: - Python: Minimum of 10 years of experience with Python programming.
- SQL: Minimum of 10 years of experience in SQL for data manipulation and querying.
- Cloud Experience: Hands-on experience with Azure cloud technologies for 5+ years (e.g., Azure Data Factory, Azure Synapse, Event Hub, AKS).
- Distributed Processing Frameworks: Experience with Kafka and Spark for distributed data processing.
Skills: - Strong communication and problem-solving skills to effectively collaborate with cross-functional teams.
- Proven ability to work on large-scale data migration projects in cloud environments.
Preferred Qualifications - API Development: Experience with GraphQL API and REST API.
- Microservices: Knowledge of data microservices architecture.
- Shell Scripting: Proficiency in Shell scripting to automate tasks and streamline processes.
Certifications Microsoft Certified: Azure Data Engineer Associate (preferred)
AWS Certified Solutions Architect (preferred)
Education: Bachelors Degree
Certification: AWS Certified Solutions Architect