Data Engineer

We are looking for a Data Engineer who can operate as a high-impact Individual Contributor with the depth and ownership of a Tech Lead. This role requires someone who can architect, build, and scale data systems end-to-end, with a strong focus on open-source and self-managed data infrastructure.

Responsibilities

• Design and build scalable, high-performance data platforms and pipelines.
• Work on distributed data systems across batch and real-time processing.
• Take end-to-end ownership from architecture to deployment and optimization.
• Debug, optimize, and extend open-source data systems.
• Solve problems at the infrastructure and systems level (not just tool usage).
• Collaborate across teams and drive data engineering best practices.
  
  

Requirements

• 5+ years of experience in Data Engineering / Data Platform roles.
• Strong understanding of distributed systems, data storage, and compute layers.
• Ability to design systems from first principles, not just use tools.
• Hands-on experience with open-source or self-managed architectures.
• Strong programming skills in Python or Go (Golang).
• Experience with system design, performance tuning, and debugging at scale.
  
  

Nice To Have

• Backend engineering experience (APIs, services, system design).
• Experience working on infrastructure and deployment.
• Exposure to high-scale or real-time systems.
  
  

Key Skills

• Query and OLAP: Trino, ClickHouse, Apache Pinot.
• Batch and Stream Processing: Apache Spark (OSS), Apache Flink (OSS).
• Table Format: Apache Iceberg.
• Cataloging: AWS Glue.
• Cloud: AWS.
• Languages: Go, Python.
  
  

This job was posted by Sneha Singh from Primathon.