data engineer in regulated financial services

The project is building a modern enterprise data and analytics platform on Microsoft Azure as part of a large-scale modernization program. It replaces a legacy policy administration system with a SaaS solution and migrates reporting and analytics from SQL Server to Databricks and Microsoft Fabric, supporting governed data products, advanced analytics, and AI-assisted engineering practices.

• Design and develop data ingestion pipelines from business systems into a governed cloud data platform;
• Build scalable data engineering solutions using Azure Databricks and Microsoft Fabric;
• Implement medallion architecture patterns and reusable engineering frameworks;
• Develop curated data products and analytical datasets for business reporting and decision making;
• Apply AI-assisted development practices while ensuring quality, governance, and auditability;
• Contribute to data modelling, semantic models, metadata management, and analytical solutions;
• Support the migration of reporting and analytics workloads from legacy platforms;
• Collaborate with stakeholders to translate business requirements into technical solutions;
• Provide design guidance, establish engineering standards, and participate in code reviews.

• 6+ Years of experience in Data Engineering;
• Hands-on experience with Azure Data Factory, Azure Databricks, and Azure Data Lake Storage Gen2;
• Strong programming experience with Python, SQL, and PySpark;
• Experience implementing Delta Lake and production lakehouse or medallion architectures;
• Experience with Git and Azure DevOps, including CI/CD pipelines for data solutions;
• Experience working with SQL Server environments;
• Practical experience using AI coding assistants within software delivery projects;
• Ability to communicate effectively in English with technical and business stakeholders;
• Experience taking ownership of architecture or platform design decisions;
• Experience reviewing code and establishing engineering standards;
• Experience gathering requirements and collaborating directly with business stakeholders;
• English for communication with technical and business stakeholders;
• Nice to have: experience with Microsoft Fabric, experience with Sapiens, Microsoft certifications including DP 203, DP 600, DP 700, AZ 900, or DP 900, Databricks Data Engineer Associate or Databricks Data Engineer Professional certification, dimensional modelling including star schema, snowflake schema, or Data Vault, semantic modelling in Power BI, knowledge of Microsoft Purview, data lineage, and metadata management, experience working in regulated financial services environments.

• UK remote, including remote London.