Senior Data Engineer in City of London

Energy Jobline is the largest and fastest growing global Energy Job Board and Energy Hub. We have an audience reach of over 7 million energy professionals, 400,000+ monthly advertised global energy and engineering jobs, and work with the leading energy companies worldwide.

We focus on the Oil & Gas, Renewables, Engineering, Power, and Nuclear markets as well as emerging technologies in EV, Battery, and Fusion. We are committed to ensuring that we offer the most exciting career opportunities from around the world for our jobseekers.

Job Description

Senior Data Engineer

London - 2 days a week in the office

Up to £95k

A high-growth digital platform is transforming a multi-billion-dollar global industry through innovative online technologies. The platform connects a large network of customers and partners worldwide, powering billions in annual transactions. With significant revenue growth and international reach, the company is modernising one of the last sectors to fully embrace digital transformation, building a global, category-defining business in the process.

As the organisation scales its AI and machine learning capabilities across search, recommendations, and analytics, it is investing heavily in robust data infrastructure to enable rapid experimentation, reliable insights, and data-driven decision-making. The Senior Data Engineer will collaborate with Product Managers, ML Engineers, Analysts, and Software Engineers to design and maintain the data pipelines and infrastructure that power AI-driven features and business intelligence, handling millions of events and requests daily.

Key Responsibilities:

• Design, build, and maintain reliable ETL/ELT pipelines to support analytics, ML models, and business intelligence
• Develop scalable batch and streaming data pipelines to process millions of events and transactions daily
• Implement workflow orchestration using Airflow, Dagster, or similar tools
• Build data validation and quality monitoring frameworks to ensure data accuracy and reliability
• Collaborate cross-functionally with Software, ML, and Analytics teams to deliver production-ready data products
• Mentor junior engineers and contribute to engineering best practices

Required Skills & Experience:

• 5+ years of experience building and maintaining data pipelines in production environments
• Strong Python and SQL skills (Pandas, PySpark, query optimisation)
• Cloud experience (AWS ) including S3, Redshift, Glue, Lambda
• Familiarity with data warehousing (Redshift, Snowflake, BigQuery)
• Experience with workflow orchestration tools (Airflow, Dagster, Prefect)
• Understanding of distributed systems, batch and streaming data (Kafka, Kinesis)
• Knowledge of IaC (Terraform, CloudFormation) and containerisation (Docker, Kubernetes)

Nice to have:

• Experience with dbt, feature stores, or ML pipeline tooling
• Familiarity with Elasticsearch or real-time analytics (Flink, Materialize)
• Exposure to eCommerce, marketplace, or transactional environments

If you are interested in applying for this job please press the Apply Button and follow the application process. Energy Jobline wishes you the very best of luck in your next career move.