Azure Data Engineer

Posted 23 September 2024
Salary £55000 - £70000 per annum
LocationCity of London
Job type Permanent
Discipline Accounting & Finance
ReferenceDB1986_1727107313
Contact NameDean Burns

Job description

Summary:

We are partnered with a fast-growing FinTech group looking to expand their data engineering team. As a Data Engineer, you will have extensive experience working with Azure cloud services, CI/CD, SQL, and Python. In this role, you will design, build, and maintain scalable data pipelines, optimise data storage solutions, and ensure data integrity and availability across their clients' platforms.

Key Responsibilities:

  • Develop, maintain, and optimise ETL processes using Azure services such as Azure Data Factory, Azure Functions, and Logic Apps.
  • Create and manage data workflows that extract data from various sources, transform it, and load it into Azure Data Lakes, Azure Synapse Analytics, or Azure SQL Databases.
  • Ensure that infrastructure is designed for scalability, security, and cost-effectiveness using Azure-native tools like Azure Monitor, Azure Security Centre, and Azure Cost Management.
  • Design and optimise data storage solutions using Azure services, including Azure Blob Storage, Azure Synapse Analytics, Azure SQL Database, and Cosmos DB.
  • Ensure data is stored efficiently and securely, with appropriate backup and disaster recovery strategies using Azure Backup, Azure Site Recovery, and Azure Key Vault.
  • Develop Python scripts to automate data processing tasks within Azure, leveraging Azure Functions, Azure Batch, or Databricks to improve workflow efficiency.

Experience:

  • 2-5 years of experience as a Data Engineer.
  • Hands-on experience with Python, Java, R (or similar).
  • Experience with cloud platforms such as Azure.
  • Extensive experience with SQL (MySQL/SQL Server).
  • Experience with data pipeline orchestration tools, such as Azure Data Factory or Apache Airflow.
  • Familiarity with CI/CD pipelines and version control systems (e.g., Azure DevOps, Git).

Benefits:

  • £55k - £70k
  • Bonus of 20-30%.
  • Opportunities for professional development and certification.
  • 3 days working from home per week.