Senior Data Engineer
Position Summary
The Senior Data Engineer owns the data platform end to end — designing, building, and maintaining the infrastructure, pipelines, and governed data layers that power analytics and downstream consumers across the organization. This is a hands-on engineering role centered on cloud data platform development, pipeline orchestration, and data modeling at scale.
The guiding principle is to build once, consume everywhere — define transformations and metrics a single time, governed and version-controlled in the warehouse, then serve them consistently to every downstream consumer: BI tools, client-facing reporting, and AI systems.
The ideal candidate is comfortable building data platforms and systems from scratch, thrive in a platform development and maintenance environment, and brings strong hands-on experience with Snowflake and workflow orchestration tools such as Apache Airflow and dbt (or equivalent platforms). This role does not own BI visualization — your work spans raw ingestion through the governed gold layer.
Key Responsibilities
Data Platform Development & Maintenance
Design, build, and maintain scalable data pipelines and platform infrastructure on Snowflake.
Architect and implement end-to-end data flows from source ingestion through the silver and gold layers.
Own platform reliability, performance tuning, and ongoing maintenance — treating the data platform as a product.
Build systems from the ground up where needed; evaluate existing systems and refactor for scale, cost efficiency, and maintainability.
Establish engineering standards: version control, CI/CD, testing frameworks, and documentation for all platform components.
Pipeline Orchestration & Workflow Management
Design and maintain orchestration workflows using Apache Airflow (or equivalent) to schedule, monitor, and manage data pipelines.
Ensure pipeline reliability, observability, and failure recovery through alerting, logging, and automated retries.
Coordinate dependencies across upstream sources and downstream consumers with well-defined SLAs.
Gold-Layer Modeling & Metrics
Design, build, and maintain gold-layer data models and a governed metrics layer in Snowflake using dbt — clean, tested, documented, and version-controlled.
Establish metric definitions as the single source of truth: calculated once, reused everywhere, with clear ownership and change management.
Push transformation and calculation logic upstream into the warehouse so the consumption layer stays simple, consistent, and high-trust.
Partner with analytics and BI stakeholders on the boundary between pipelines (data engineering) and consumption (analytics/reporting).
Structure and document data models so they meet the governance and consistency requirements of AI agents and autonomous analytical systems.
AI-Native Development
Build with coding agents and AI-assisted tooling as a primary part of the workflow — accelerating pipeline development, data modeling, documentation, and testing.
Independently set up and manage development environments, repositories, and the tooling needed to work effectively with agentic systems.
Help establish team practices and standards for building data infrastructure in the AI era.
Quality, Governance & Platform Judgment
Champion data quality, consistency, and trust across all platform layers — ensuring data holds up under scrutiny from any downstream consumer.
Contribute to platform architecture decisions; identify opportunities for automation, templating, and process improvement.
Apply sound engineering judgment to evaluate build vs. buy decisions and tooling trade-offs.
Qualifications
Required
5+ years of hands-on experience in data engineering — building pipelines, data models, and platform infrastructure in a data-driven organization.
Snowflake: Proven, hands-on production experience building data models and transformations in Snowflake. Deep familiarity with Snowflake architecture, performance optimization, and cost management.
Workflow Orchestration (must have): Hands-on experience with Apache Airflow or an equivalent DAG-based orchestration platform (e.g., Prefect, Dagster, Mage). Ability to design, deploy, and operate complex pipeline workflows.
dbt: Demonstrated experience using dbt for transformation modeling, testing, and documentation in a production warehouse environment. Equivalent semantic layer tooling experience considered.
Strong SQL proficiency — able to write, optimize, and review complex queries across large datasets.
Experience building data platforms or systems from scratch — not just extending existing infrastructure but designing and standing up new ones.
Comfort with platform development and maintenance as a long-term ownership model — not just project delivery.
Fluency working in the AI era: building with coding agents, setting up development environments, and using AI tooling as a core productivity multiplier.
Familiarity with software engineering practices: version control (Git), CI/CD pipelines, automated testing, and code review.
Strong communication skills — able to translate business requirements into technical data platform decisions.
Preferred
Experience in pharmacy benefits management, healthcare technology, or a related regulated industry.
Experience with cloud infrastructure and data platform tooling (AWS, Azure, or GCP) alongside the data stack.
Familiarity with data governance frameworks, data contracts, or data mesh principles.
Experience with streaming or near-real-time data pipelines (e.g., Kafka, Kinesis, Spark Streaming).
Exposure to semantic/metrics layer tooling (e.g., MetricFlow, Cube, LookML).
Experience building demo environments or client-facing data products.
Salary Range: 145,000 - 165,000