Databricks Systems Engineer

Job DescriptionJob DescriptionApply now: Databricks System Engineer , location is Remote . The start date is ASAP for this Contract position.

Job Title: Databricks Systems Engineer
Location-Type: Remote
Start Date Is: ASAP
Duration: Contract 5.5 months
Compensation Range: $49.50-$65.00/hr W2

Responsibilities :

• Architect, configure, and optimize Databricks Pipelines for large-scale data processing within an Azure Data Lakehouse environment.
• Set up and manage Azure infrastructure components including Databricks Workspaces, Azure Containers (AKS/ACI), Storage Accounts, and Networking.
• Design and implement a monitoring and observability framework using tools like Azure Monitor, Log Analytics, and Prometheus/Grafana.
• Collaborate with platform and data engineering teams to enable microservices-based architecture for scalable and modular data solutions.
• Drive automation and CI/CD practices using Terraform, ARM templates, and GitHub Actions/Azure DevOps.

Requirements :

• Strong hands-on experience with Azure Databricks, Delta Lake, and Apache Spark.
• Deep understanding of Azure services: Resource Manager, AKS, ACR, Key Vault, and Networking.
• Proven experience in microservices architecture and container orchestration.
• Expertise in infrastructure-as-code, scripting (Python, Bash), and DevOps tooling.
• Familiarity with data governance, security, and cost optimization in cloud environments.

Nice to have :

• Experience with event-driven architectures (Kafka/Event Grid).
• Knowledge of data mesh principles and distributed data ownership.