We are looking for a Senior Databricks Data Engineer to join our team.
This role is ideal for someone who enjoys building end-to-end solutions and wants to work on real-world industrial use cases that comes from an Engineering background, and has experience in Oil & Gas.
In this position, you’ll contribute to a variety of impactful client projects, including:
- Building AI-powered anomaly detection systems for operational and industrial data
- Modernizing asset management workflows
- Developing natural language interfaces for querying enterprise and operational data
- Implementing Azure Databricks foundations and pipelines with best practices
- Creating scalable data pipelines and ML workflows in cloud environments
- Enabling real-time and batch data processing for analytics and AI use cases
- Working through complex and challenging data conditions, including incremental processing, late-arriving or changing records, complex business and engineering rules, and reconciliation of results across processing runs
The ideal candidate combines strong data engineering experience with an engineering or applied-science background. Direct experience industrial time-series data, scientific measurements, telemetry, financial reconciliation, or other datasets where accuracy, traceability, and incremental recalculation are critical.
Key Responsibilities
- Design, build, and optimize pipelines for sensor and related operational data.
- Develop complex transformation and calculation logic based on engineering requirements.
- Implement robust incremental-processing patterns for high-volume and continuously changing datasets.
- Design, build, and deploy end-to-end AI/ML solutions in production environments
- Develop robust backend systems and APIs to support AI-driven applications
- Build and maintain data pipelines and feature engineering workflows
- Implement and operationalize machine learning models (training, deployment, monitoring)
- Work with modern AI tooling (LLMs, agents, orchestration frameworks)
- Collaborate with clients to translate business problems into technical solutions
- Contribute to architecture decisions and best practices across projects
- Mentor client team members and contribute to internal capability building
- Work directly with engineering and operational SMEs to understand physical processes and translate their knowledge into technical requirements.
- Make engineering calculations and data transformations explainable, traceable, testable, and auditable.
- Document data lineage, calculation logic, assumptions, dependencies, and exception-handling rules.
Ideal Background
- Senior-level experience designing and developing production data pipelines with Azure Databricks including strong experience with complex SQL, Python, Spark, or comparable data-processing technologies.
- Complex Excel and CSV integration experience
- Experience with Databricks dashboards and Genie
- Strong requirements gathering experience with Field Engineers and Technical SMEs
- Demonstrated experience with incremental processing, change detection, reconciliation, and pipeline design
- Experience handling time-series, telemetry, sensor, operational, scientific or industrial data
- Medallion architecture and modelling
- Strong analytical and investigative skills, with the patience to work through detailed logic and difficult data-quality problems.
- Degree or professional background in petroleum, reservoir, chemical, mechanical, geological, geophysical, or another relevant engineering or applied-science discipline is strongly preferred.
- Experience in upstream oil and gas, thermal operations, SAGD, well surveillance, production engineering, or subsurface data would be a significant asset.
This role presents an exciting opportunity to work on practical, high-impact AI use cases - not just prototypes, shape how AI is applied to client environments, and change the game on traditional processes and platforms. Come join a growing organization helping clients take a new, lean and value-driven approach to data and engineering!
