Lead Cloud Data Engineer - Remote Opportunity in Duluth

Energy Jobline is the largest and fastest growing global Energy Job Board and Energy Hub. We have an audience reach of over 7 million energy professionals, 400,000+ monthly advertised global energy and engineering jobs, and work with the leading energy companies worldwide.

We focus on the Oil & Gas, Renewables, Engineering, Power, and Nuclear markets as well as emerging technologies in EV, Battery, and Fusion. We are committed to ensuring that we offer the most exciting career opportunities from around the world for our jobseekers.

Job DescriptionJob Description

We are building a next- Cloud Data Platform to unify data from Policy, Claims, Billing, and Administration systems into a single source of truth. We are seeking a Lead Cloud Data Engineer who will be 75% hand-on and play a critical role in designing, building, and optimizing our modern data ecosystem leveraging Medallion architecture, Delta Lake, and modern data warehouse technologies such as Snowflake, Synapse, or Redshift.

As a technical leader, the Lead Data Engineer will define and execute the end-to-end data engineering strategy from data ingestion and modeling to governance and performance optimization enabling scalable, high-quality, and analytics-ready data assets. This role requires hands on deep technical expertise in cloud- data engineering, automation, and architecture design, coupled with strong leadership to mentor teams and align solutions with business goals.

Responsibilities:

Data Platform Design & Architecture

• Define the strategic roadmap for the enterprise data platform, ensuring scalability, performance, and interoperability across business domains.
• Architect and implement cloud-, Medallion-based data architectures (Bronze–Silver–Gold layers) for unified and governed data delivery.
• Drive standardization of data models, pipelines, and quality frameworks across Policy, Claims, Billing, and Administrative data assets.
• Evaluate and implement emerging data technologies to strengthen the platform’s performance, cost efficiency, and resilience

Data Integration & Ingestion

• Design, build, and optimize high-performance ingestion pipelines, using AWS Glue, Databricks, or custom Spark applications.
• Automate ingestion of structured, semi-structured, and unstructured data from APIs databases, and external data feeds.
• Tune and monitor ingestion pipelines for throughput, cost control, and reliability across dev/test/prod environments.

Data Transformation & Modeling

• Hands on Development of ETL/ELT pipelines using Databricks or similar frameworks to transform raw data into curated and consumption-ready datasets.
• Design and develop relational, Vault, and dimensional data models to support analytics, BI, and AI/ML workloads.
• Define and enforce data quality standards, validation frameworks, and enrichment rules to ensure trusted business data.
• Apply data quality, cleansing, and enrichment logic to ensure accuracy and completeness of business-critical data.

Cloud Infrastructure, Automation and Performance

• Collaborate with DevOps and Cloud Engineering teams to design automated, infrastructure-as-code environments using Terraform, CloudFormation, or equivalent tools.
• Implement CI/CD pipelines for data pipeline deployment, versioning, and testing.
• Lead performance tuning and scalability optimization to ensure highly available, cost-efficient data platform.

Governance, Security & Compliance

• Implement and enforce data governance, cataloging, and lineage practices using tools such as Purview, Alation, or Collibra.
• Partner with InfoSec to implement data privacy, access control, and compliance frameworks aligned with regulatory standards.
• Drive consistency and accountability in data stewardship across business and IT teams.

Leadership, Collaboration & Mentorship

• Lead a team of data engineers, providing technical guidance, coaching, and performance mentorship.
• Collaborate with Data Architects, Analysts, and Business Leaders to align data solutions with enterprise strategy.
• Promote a culture of engineering excellence, reusability, and knowledge sharing across data organization.
• Influence enterprise-wide standards for data engineering, automation, and governance.

Qualifications:

• Bachelor’s or master’s degree in computer science, Data Engineering, or a related field.
• 12+ years of experience in data engineering with at 3+ years in a lead or architect-level role and least 8+ years on cloud platforms (AWS, Azure, or GCP).
• Deep hands-on experience with Python, SQL, and data modeling (relational, and Dimensional), Databricks, Spark, AWS Glue, Delta Lake, Snowflake, Synapse, or Redshift
• Proven experience with Medallion architecture, modern data warehousing principles., data governance, lineage, and CI/CD for data pipelines
• Excellent leadership, communication, and cross-functional collaboration skills.
• Experience in the Property & Casualty (P&C) Insurance domain such as Policy, Claims, or Billing data .
• Familiarity with event-driven architectures (Kafka, Kinesis) and real-time data streaming.
• Knowledge of machine learning pipeline integration and feature engineering.
• Proven ability to lead large-scale data modernization or cloud migration initiatives.

Compensation:

• $140,000 - $165,000 commensurate with experience, plus bonus eligibility

Benefits:

We are proud to offer a robust benefits suite that includes:

• Competitive base salary plus incentive plans for eligible team members
• 401(K) retirement plan that includes a company match of up to 6% of your eligible salary
• Free basic life and AD&D, long-term and short-term insurance
• Medical, dental and vision plans to meet your unique healthcare needs
• Wellness incentives
• Generous time off program that includes personal, holiday and volunteer paid time off
• Flexible work schedules and hybrid/remote options for eligible positions
• Educational assistance

#TMG

If you are interested in applying for this job please press the Apply Button and follow the application process. Energy Jobline wishes you the very best of luck in your next career move.