Job Summary: As a Sr. Data Engineer, you will create and maintain optimal data pipeline architecture, Infrastucture automation(IAC) and also will assemble large, complex data sets that meet functional/non-functional business requirements across Data Solutions.
You will also identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability etc.
Implement foundational and advanced analytics methodologies to address key metrics to improve customer experience, network performance, quality of service, profitability of products, and personalized offerings
Delivering technical functionality developed against the Global Lockton Cloud Data Platform's with a Product Mindset.
Use data, analysis, and automation to generate value within the company through advanced data modeling and other advanced analytics principles
Building the infrastructure required for optimal ETL/ELT of data from a wide variety of data sources using Cloud Technologies and OnPrem.
Responsible for helping an organization deliver high data availability and quality throughout the entire data life cycle from ingestion to end products: dashboards, machine learning models, and production datasets.
The Successful Applicant
Technical Knowledge, Key skills and experience required for the successful candidate:
Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, ADLS-Delta lake, Azure functions, Logic aps, Synapse, Azure Logic Apps, Azure Storage, Azure Search, Purview, Azure SQL server, Scala/Spark/SQL/PYTHON etc.
Azure Database development experience with Infra as a Code, Azure CI/CD, Azure Classic Pipelines, Azure YAML, ARM/Terraforms/Ansible, AKS Maintenance, Azure ML, Pipeline integrations, Azure PaaS and IaaS, Azure Infra Experience (Spinning VM's, Databases and Cosmos)
Expertise in Data warehousing - data modelling, relational and Dimension modelling, Data quality, data catalog and Data Governance
Design and build highly scalable data pipelines using new generation tools and technologies like Spark, Kafka to induct data from various systems
Experience with data security and data access controls and design across digital transformation/cloud migration programs.
Deep understanding of relational as well as NoSQL data stores, methods and approaches (star and snowflake, dimensional modelling)
Ensure proper requirement traceability and Quality delivery during development and integration testing
A strong background in the Microsoft SQL Server stack (including SSIS, SSRS, Power BI, and SSAS).
Ability to build processes supporting data transformation, data structures, metadata, dependency, and workload management.
Strong analytic skills related to working with structured and unstructured datasets
Should support the Production live application by maintaining 100% Data accuracy and data availability in all 24 7.
Requirements: - Minimum of 7+ years of increasingly responsible experience in Data Platforms, Data Engineering, Software Engineering at high impactful and changing environments
- 4+ years of working experience in Azure or cloud platforms as Solution/Data Architect/Data Engineer
- 4+ years of hands-on experience with one or more cloud technologies such as Azure, AWS, Google.
- Experience in collaborating with a high-profile team of Data Engineers, data scientists, Business Analytics and Data Analysts
- Knowledge of Software development agile methodologies and worked in Scrum teams
- Excellent communication & presentation skills with a track record of engaging with senior business leaders
- Experience in Finance/Insurance would be a distinct advantage
- Should have worked in a dynamic and fast-changing environment
- High affinity with AI-powered insight tools and engines and application of real-time solutions to analytics problems
- Experience with GEN AI and LLM's will be added advantage
- Exposure to COGS/KTLO, Operations and Command Center activity
Soft Skills Required
- Good communication and presentation skills
- Highly driven, energetic, flexible, resourceful & ability to multitask
- Clarity of thoughts and vision
- Ability to ideate and bring solutions to the table
- Adherence to timelines, without sacrificing the quality of output
- Hands-on and detail-oriented, with a strong ability to coordinate across different Geographies and with different stakeholders
Academics
- Graduate degree in Engineering, Computer Science, Management Information Systems, or related
- Strong exposure to Databases and data structures of cloud-based technologies
- Azure/GCP/AWS Solution Architect certification or Data Engineer certification is plus
Career progression:
- Receive ongoing support and funding towards your personal learning and development, including coverage for relevant courses and certifications in data related fields.
- Lockton prides itself on offering outstanding career progression programmes for those who wish to progress in their field of interest, be it in a Technical or Leadership capacity