Senior Data Engineer (eDV)

Due to the nature of the work and the customers, you must hold enhanced DV Security Clearance.

As a Data Engineer, you will assist in the development of a secure batch/real-time data platform for a government client leveraging the latest commercial and open-source technologies. This involves getting data from a variety of different sources, getting it into the right format, assuring alignment to target models, and ensuring that data can be made available to downstream consumers.

Duties & Responsibilities

Code, test, and document new or modified data pipelines that meet functional / non- functional business requirements

Conduct logical and physical database design

Expand and grows data platform capabilities to solve new data and analytics problems

Conduct data analysis, identifying feasible solutions and enhancements to data

processing challenges

Ensure that data models are consistent with the data architecture (e.g. entity names,

relationships and definitions)

Perform root cause analysis on internal and external data and processes to answer

specific business questions and identify opportunities for improvement

Core Experience & Competencies

The successful candidate will have a Bachelor's degree and meet all of the following essential criteria and additionally have some experience (or a genuine interest in learning) from the preferred criteria.

Essential

Data pipeline development using data processing technologies and frameworks

Agile or other rapid application development methods

Data modelling and understanding of different data structures and their benefits and limitations under particular use cases

Experience in Public Cloud services, such as AWS. Practical experience with core services such as EC2, RDS, Lambda, Athena & Glue would be even better!

Configuring and tuning Relational and NoSQL databases, including both query processing and query planning, or other data processing infrastructure

Programming or scripting languages, such as Python

Test Driven Development with appropriate tools and frameworks

Preferred

Familiar with CI tools such as Jenkins/Drone

Knowledge of configuration tools such as Ansible & Packer

Use / Knowledge of Terraform for IaC deployments

Experience in Linux and Windows