The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
The ad says“Strong understanding of distributed storage systems, partitioning strategies, and file formats such as Parquet, Avro, and ORC”
The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
The ad says“Experience with Docker, Kubernetes, and modern containerization technologies”
The ad says“Strong proficiency in Python (PySpark), Java, Scala, or similar programming languages”
The ad says“Experience with Docker, Kubernetes, and modern containerization technologies”
The ad says“Strong understanding of distributed storage systems, partitioning strategies, and file formats such as Parquet, Avro, and ORC”
The ad says“Strong understanding of distributed storage systems, partitioning strategies, and file formats such as Parquet, Avro, and ORC”
The ad says“Strong proficiency in Python (PySpark), Java, Scala, or similar programming languages”
The ad says“Strong proficiency in Python (PySpark), Java, Scala, or similar programming languages”
The ad says“Strong proficiency in Python (PySpark), Java, Scala, or similar programming languages”
The ad says“Advanced SQL expertise, including performance tuning and optimization across large datasets”
Values
The ad says“Experience using AI-assisted coding tools (e.g., GitHub Copilot, Claude Code) to accelerate development while maintaining quality is encouraged”
The ad says“Experience with orchestration and workflow management tools such as Argo, Airflow, or similar technologies”
The ad says“Experience with orchestration and workflow management tools such as Argo, Airflow, or similar technologies”
The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
The ad says“Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)”
As a Staff Data Engineer on the Data Lake team at H1, you will play a critical role in shaping the architecture, scalability, reliability, and long-term direction of our core data platform. This role is designed for a highly technical engineer who is excited to grow into an Engineering Manager track while remaining deeply hands-on technically
The Data Lake is the foundation of H1’s platform, responsible for the validation, accuracy, standardization, and quality of the data powering every downstream product and team across the organization. You will help lead the evolution of this platform while supporting and mentoring a growing team of engineers
Architect, build, and scale distributed ETL/ELT pipelines and large-scale ingestion frameworks across structured and unstructured healthcare datasets
Lead the evolution of H1’s Data Lake architecture with a focus on scalability, observability, reliability, and cost optimization
Own and improve data quality, validation, normalization, and standardization workflows across thousands of global data sources
Design and optimize batch and near real-time data processing frameworks using cloud-native distributed systems
Optimize distributed compute and storage systems, including Spark workloads, query performance, partitioning strategies, and infrastructure efficiency
Drive improvements in monitoring, governance, operational excellence, and production reliability across the platform
Troubleshoot complex production data and infrastructure issues across distributed systems
Partner closely with Product, Infrastructure, Security, Compliance, and downstream engineering teams to support scalable and secure data delivery
Mentor engineers through technical leadership, architecture reviews, and engineering best practices
Help define technical roadmap priorities and contribute to long-term platform strategy and execution planning
Support production operations, incident response, and platform health as part of overall ownership of the Data Lake ecosystem
Benefits
Flexible work hours
Commuter benefits
Stock options
Computer setup
Work from home opportunities
Health & life insurance
Retirement options
Unlimited PTO
Flex Give & Flex Spend
Impactful BRGs
Preferred experience
You are a highly technical data engineer who thrives in lean, high-ownership environments and enjoys solving complex distributed systems challenges. You are excited by the opportunity to influence technical direction, mentor engineers, and grow into broader engineering leadership responsibilities while remaining hands-on
Strong proficiency in Python (PySpark), Java, Scala, or similar programming languages
Experience using AI-assisted coding tools (e.g., GitHub Copilot, Claude Code) to accelerate development while maintaining quality is encouraged
Experience with Docker, Kubernetes, and modern containerization technologies
Experience designing and scaling modern cloud-native data lake architectures and large-scale ingestion frameworks
You excel at building reliable, observable, and maintainable data systems supporting critical business and analytics workloads
Deep experience with Apache Spark and cloud-native big data platforms, preferably within AWS environments (EMR, Glue, S3, Athena, Redshift, or similar)
Experience with large-scale data cleaning, parsing, normalization, and validation workflows preferred
Experience with orchestration and workflow management tools such as Argo, Airflow, or similar technologies
You are energized by ownership, autonomy, and solving ambiguous technical challenges
You have deep experience designing and scaling distributed data platforms and large-scale pipelines in cloud-native environments
Exposure to ML/AI-driven data enrichment, parsing, or validation workflows is a plus
You are comfortable leading technical initiatives and influencing architecture decisions across teams
8+ years of experience in data engineering, software engineering, or related fields with significant experience building and scaling distributed data platforms
You have strong expertise in distributed processing, performance optimization, and modern data architecture patterns
Strong understanding of distributed storage systems, partitioning strategies, and file formats such as Parquet, Avro, and ORC
Demonstrated technical leadership experience with interest in or experience mentoring and leading engineers
You communicate effectively with both technical and non-technical stakeholders
Experience working with healthcare, life sciences, publication, or large-scale entity-resolution datasets preferred
Experience implementing monitoring, observability, and data quality frameworks within production environments
You enjoy mentoring engineers and helping raise the engineering bar across teams
Advanced SQL expertise, including performance tuning and optimization across large datasets
