Senior Data Engineer

Senior Data Engineer

AI Engineering  |  Forward Deployed Engineering

Role Description

Defense Unicorns is seeking a senior Data Engineer to embed with a government team and its technology partners building an emerging AI-powered engineering ecosystem.

This role will focus on making enterprise data usable, discoverable, governed, and accessible to AI models, agents, and mission applications. The engineer will work across data ingestion, transformation, storage, metadata, APIs, search, vector and graph technologies, and the interfaces between the data layer and AI workloads.

The ideal candidate is a hands-on data engineer who can operate across the full data lifecycle and understands that data availability and integration are often the primary blockers to delivering AI capabilities. They should be comfortable entering an evolving architecture, learning unfamiliar systems quickly, working directly with commercial technology partners, and solving difficult problems involving data quality, lineage, access, performance, and security.

This is not a traditional analytics or business intelligence role. The focus is on building the data foundations and repeatable patterns that allow AI and agentic capabilities to access trusted data and move from prototype to production in secure and classified environments.

Responsibilities

Design, build, and maintain reliable data pipelines that ingest, transform, validate, and deliver data for AI models, agents, and mission applications.

Integrate structured and unstructured data sources into enterprise data services, including relational, document, vector, and graph-oriented technologies.

Work closely with AI/agent engineers to make data discoverable and usable for RAG, semantic search, model context, and tool-enabled agent workflows.

Develop repeatable data ingestion and transformation patterns using Python, SQL, APIs, event-driven architectures, and workflow/orchestration technologies.

Establish data quality, validation, provenance, lineage, and metadata practices so downstream users and AI systems can trust the information they consume.

Design data access patterns that enforce appropriate identity, authorization, classification, and security boundaries while minimizing unnecessary friction for developers and mission users.

Collaborate with platform engineers to operationalize data services within Kubernetes/OpenShift environments and integrate them into CI/CD and GitOps workflows.

Help integrate enterprise data capabilities such as relational stores, JSON/document data, vector search, graph data, advanced analytics, and RAG services into the broader platform architecture.

Troubleshoot end-to-end data issues spanning source systems, pipelines, storage, APIs, identity, networking, platform services, and AI applications.

Work directly with government engineers and commercial technology partners to resolve data dependencies and remove blockers to mission delivery.

Develop documentation, data contracts, schemas, architecture decision records, runbooks, and operational standards that make successful patterns repeatable across environments.

Identify recurring data engineering challenges that can be standardized, automated, or productized rather than repeatedly solved through manual engineering.

Operate effectively in a fast-moving, ambiguous environment where data sources, architectures, security requirements, and mission priorities will continue to evolve.

Minimum Experience and Qualifications

Active TS/SCI clearance.

7+ years of experience in data engineering, data platform engineering, or a closely related field.

Strong proficiency with SQL and Python for data processing, automation, and integration.

Hands-on experience building and operating production data pipelines and ETL/ELT workflows.

Experience with relational databases and modern data storage patterns for structured and unstructured data.

Experience integrating data through REST APIs, message/event systems, or other distributed integration patterns.

Working knowledge of data modeling, schema design, data quality, lineage, metadata, and data governance concepts.

Experience operating in cloud-native or containerized environments and collaborating with Kubernetes/platform engineering teams.

Ability to troubleshoot data issues across infrastructure, applications, APIs, storage, networking, and access controls.

Ability to work independently, rapidly learn new technologies, and translate ambiguous mission needs into working data solutions.

Local to the National Capital Region and able to support onsite work at a government facility in Springfield, VA as mission needs increase.

Preferred Experience and Qualifications

Experience supporting AI/ML workloads, including RAG, embeddings, vector search, model context, or agentic applications.

Experience with vector databases, graph databases, document databases, knowledge graphs, or semantic search technologies.

Experience with Oracle or comparable enterprise data platforms supporting relational, JSON/document, vector, graph, and analytics workloads.

Experience designing data products or platforms for high-scale, low-latency, or mission-critical workloads.

Experience with data orchestration and workflow tools such as Airflow, Dagster, Argo Workflows, or comparable technologies.

Experience with streaming/event platforms such as Kafka, Redpanda, NATS, or similar technologies.

Experience deploying data services using Kubernetes, Helm, GitOps, Infrastructure as Code, and automated CI/CD pipelines.

Experience supporting data and AI systems in Secret or TS/SCI environments and across multiple security domains.

Experience with disconnected or air-gapped environments and the challenges of moving data, metadata, and software across security boundaries.

Familiarity with NIST 800-53, RMF/ATO processes, zero-trust architectures, and data security/compliance practices.

Experience with enterprise search, RAG pipelines, data catalogs, data contracts, or knowledge-graph architectures.

Familiarity with UDS, Zarf, Pepr, Iron Bank, or similar secure software delivery technologies.

Experience working on multidisciplinary teams consisting of government engineers, FDEs, OEM professional services teams, and multiple technology vendors.

Demonstrated ability to take a fragmented or difficult data environment and turn it into a reliable, reusable foundation for mission applications and AI.

Mission Focus

Make trusted data available where AI needs it, while establishing the repeatable, secure data patterns required to move from experimentation to operational capability.