Required Skills: Very good knowledge in working with Azure Databricks services for data engineer
- Creating Azure Databricks services
- Reading & writing data from/to Azure Blob storage & Azure Synapse Gen2 storage using Azure Databricks services after receiving in different formats (e.g. csv, Parquet, Jason etc.) from Snowflake DWH platform
- Hands-on experience in working with Databricks File system (DBFS)
- Experience in working with RDD, Dataframe, Azure SQL database
- Experience in constructing ingestion, transformation & egree pipelines using Azure Data Factory
- Experience in developing application service using Pyspark & Spark SQL