Senior Data Engineer

Purpose:

We are looking for a savvy Senior Data Engineer to join our growing team of Data experts. The hire will be responsible for expanding and optimizing our data pipeline architecture and data flow and collection for cross-functional teams and maintaining governance over our lakehouse. The Data Engineering team will partner with fellow software engineers to build data pipelines and solve data-related problems within the company.

Responsibilities:

Develop, deploy, and maintain Big Data solutions that will ingest, process, and store the necessary data to power Talkdesk’s business

Design batch or streaming dataflows capable of processing large quantities of fast-moving unstructured data

Monitoring dataflows and underlying systems, promoting the necessary changes to ensure scalable, reliable, and high-performance solutions

Work closely with the rest of Talkdesk’s engineering to deliver world-class data-driven solutions

Required Skills:

Strong understanding of distributed computing principles and distributed systems

Proven experience in building datalake and/or lakehouse data architectures, integrating large volumes of data sourced from different sources predominantly in a real-time fashion

Proficient in one or more languages like Java, Kotlin, or Scala. Python developers willing to work on JVM are also welcome

Experience with messaging systems, such as Kafka, RabbitMQ, or ActiveMQ

Experience with distributed processing engines, such as Flink, Spark, and/or Kafka Streams

Good knowledge of analytical tools such as Trino, Dremio, Impala or similar

Experience with table formats such as Hive, Iceberg, Delta Lake, Hudi or similar

Experience with relational databases, including data modeling, scalability strategies, and performance analysis

Experience with cloud environments such as AWS, Azure, or Google Cloud

At least 4 years of relevant professional experience

Strong written and verbal English communication skills

Nice to have / Pluses:

Experience with NoSQL databases, such as MongoDB, Cassandra, ElasticSearch, or Redis;

BS/MS Degree in Computer Engineering, Computer Science, Applied Math, or another engineering-related field

Experience in Agile development methodology/Scrum

Good understanding of Lambda and Kappa Architectures, along with their advantages and drawbacks

Experience in containerization and orchestration technologies, including Docker and Kubernetes, to streamline application deployment and management.