Senior Data Engineer
Senior Data Engineer (dbt / Databricks Migration)
.Monks Technology Services, part of Media.Monks and S4 Capital, is a global consulting firm mastering AI-powered transformations for the Fortune 100. We combine long-term strategic thinking, deep enterprise experience, and a human-centered approach to help clients transform business processes and dominate their industries.
As part of our commitment to inclusion and in compliance with Law 2466 of 2025 from Colombia, we have available spaces for people with disabilities.
Fixed-term contract
About the Role
Monks is seeking a Senior Data Engineer to join a collaborative engineering pod delivering a data platform modernization project, migrating a legacy Python/Spark ETL framework into dbt Core models on Databricks (Delta Lake, Unity Catalog). You will help modernize a large set of production reporting tables that power analytics, product, and BI functions.
In this role, you will work alongside a Lead Data Engineer and an internal client engineer to handle the end-to-end migration of assigned legacy ETL configurations. You will move beyond maintenance to reverse-engineer legacy logic, author maintainable dbt SQL models, enforce strict testing protocols, and validate data output parity in a structured, documentation-first, PR-reviewed environment.
Key Responsibilities
ETL Migration & Modeling: Reverse-engineer existing Python/Spark ETL logic and author corresponding dbt SQL models, staging layers, and foundational shared macros for dimension and domain models.
Quality Assurance & Testing: Write comprehensive schema.yml tests (not_null, unique, relationships) and lead output-parity validations to guarantee newly built models produce output consistent with legacy production pipelines before go-live.
CI/CD & Delivery: Ship changes through a Git-based, PR-reviewed workflow utilizing automated CI/CD gates (dbt compile + dbt test on every PR).
Platform Collaboration: Work natively within Databricks using Delta Lake, Unity Catalog, and Databricks SQL Warehouses in close alignment with internal client engineers.
Cross-Functional Partnership: Operate as an embedded partner within a collaborative engineering pod, ensuring seamless integration, knowledge sharing, and adherence to modern data engineering best practices.
About You
Competencies
Ownership & Attention to Detail: Rigorous about output parity and data quality, taking full responsibility for models from legacy conversion through production release.
Collaborative Mindset: Thrives in an embedded, partnership-driven engagement alongside client engineers rather than traditional handoff setups.
Engineering Discipline: Strong commitment to software engineering best practices, documentation-first development, and automated testing workflows.
Problem-Solving & Adaptability: Skillful at untangling fragile, hardcoded SQL/Python logic and refactoring it into modular, maintainable architectures.
Qualifications & Skills
dbt Core Expertise: 2+ years of hands-on production experience. Comfortable with Jinja macros, sources.yml, schema.yml tests, and incremental model patterns (dbt Certified Developer preferred).
Databricks Ecosystem: Proven hands-on experience with Delta Lake, Unity Catalog, and Databricks SQL Warehouses. Exposure to Databricks Workflows for scheduled runs is a plus.
Advanced SQL & Legacy Migration: Exceptional skills with CTEs, window functions, and converting custom/legacy Python-based ETL frameworks into modular, source-agnostic dbt SQL models.
Git & CI/CD Workflows: Demonstrated experience working in PR-reviewed environments with automated CI/CD testing gates (e.g., GitHub Actions).
Validation & Domain Experience (Nice to Have): Experience validating output parity against production data. Familiarity with third-party connectors (e.g., Fivetran), media/e-commerce analytics, or async-first remote delivery is a plus.
.Monks Technology Services does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment.
#LI-Remote
#LI-ES1