TixelJobs
E
Effectualvia Greenhouse

Senior Data Engineer

REMOTE$150K - $180K/yrPosted 5mo ago
Data EngineerSeniorFull-time

About the Role

Position Summary 

We are seeking a Senior Data Engineer with specialized expertise in data streaming technologies to join our data team. This role focuses on building and maintaining high-performance data streaming architectures that enable real-time data processing and analytics. The ideal candidate will have deep experience with Apache Kafka, AWS Managed Streaming for Apache Kafka (MSK), Amazon Kinesis, and related streaming technologies in cloud environments. 

Role Focus 

A Senior Data Engineer at Effectual is primarily responsible for building and maintaining the streaming data architecture that enables real-time data processing and analytics. This involves constructing robust data streaming pipelines that transform and transport data from various sources in real-time, ensuring data flows efficiently through streaming systems for immediate analysis and operational decision-making. You will focus on the efficient and secure management of streaming data systems, ensuring that data is processed, stored, and made available for real-time analytics and downstream applications. 

Essential Duties and Responsibilities  

Streaming Data Architecture & Pipeline Development 

  • Design, build, and maintain scalable streaming data architectures using Kafka, MSK, and Kinesis 
  • Develop real-time data pipelines that handle high-volume, high-velocity data streams 
  • Implement event-driven architectures and microservices patterns for streaming data processing 
  • Create and optimize data streaming topologies for complex event processing scenarios 
  • Design fault-tolerant streaming systems with proper error handling and data recovery mechanisms 

Kafka & MSK Management 

  • Configure, deploy, and manage Apache Kafka clusters and AWS MSK environments 
  • Implement Kafka Connect pipelines for streaming data integration 
  • Share