Middle Big Data Engineer
We are seeking a proactive Middle Big Data Engineer to join our team. In this role, you will be responsible for designing, developing, and maintaining scalable data pipelines and reporting-ready data assets within Palantir Foundry. The ideal candidate will possess a robust background in cloud technologies, data architecture, and a passion for solving complex data challenges.
You will collaborate with a cross-functional team and stakeholders to translate business requirements into scalable data solutions, enabling reporting, improving data quality, and driving data transformation initiatives.
About the Client:
Our client is a leading global healthcare organization, driving innovation in pharmaceuticals, biotechnology, and patient-centric solutions. They develop advanced data and analytics solutions that help global organizations turn complex data into intuitive digital tools, enabling teams to make informed decisions in their daily operations and long-term strategy. Their projects span modern operational applications, interactive dashboards, and advanced analytics platforms designed to deliver actionable insights across multiple business domains.
Key Responsibilities:
Collaborate with cross-functional teams to gather business requirements and design, implement, and maintain scalable data pipelines in Palantir Foundry, ensuring end-to-end data integrity and optimized workflows
Design and maintain data ingestion, transformation, and orchestration pipelines using Palantir Foundry, Python, and PySpark
Develop reporting-ready datasets and data models to support downstream analytics and Power BI reporting
Develop, optimize, and maintain efficient ETL/ELT processes to collect, process, and integrate data from multiple sources, ensuring timely and accurate data delivery
Implement data quality validation, monitoring, and health checks to ensure reliability and consistency of data assets
Monitor pipeline performance, identify bottlenecks, troubleshoot production issues, and continuously improve scalability and processing efficiency
Plan and prioritize work independently, communicate progress transparently, and proactively manage changing priorities while proposing practical solutions when requirements evolve
Maintain clear technical documentation and promote automation and continuous improvement across data engineering processes
Stay current with emerging technologies and industry best practices, incorporating innovative approaches into data engineering solutions
Required Qualifications:
3+ years of experience in data engineering, preferably within the pharmaceutical or life sciences industry
Strong proficiency in Python and PySpark
Proficiency with big data technologies (e.g., Apache Hadoop, Spark, Kafka, BigQuery, etc.)
Hands-on experience with cloud services (e.g., AWS Glue, Azure Data Factory, Google Cloud Dataflow)
Expertise in data modeling, data warehousing, and ETL/ELT concepts
Hands-on experience with database systems (e.g., PostgreSQL, MySQL, NoSQL, etc.)
Proficiency in containerization technologies (e.g., Docker, Kubernetes)
Effective problem-solving and analytical skills, coupled with excellent communication and collaboration abilities
Nice to have:
Experience with market research data
Familiarity with Veeva CRM, Reltio, SAP, and/or Palantir Foundry
Background in pharma or healthcare
Experience with automation / pipeline optimization
We offer*:
Flexible working format - remote, office-based or flexible
A competitive salary and good compensation package
Personalized career growth
Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
Active tech communities with regular knowledge sharing
Education reimbursement
Memorable anniversary presents
Corporate events and team buildings
Other location-specific benefits
*not applicable for freelancers