Data Engineer

Data Integration & Pipeline Support

• Support and enhance ETL and ELT pipelines
• Integrate data from diverse sources including databases, APIs, filesystems, streaming platforms, and cloud storage.
• Troubleshoot data flow failures, latency issues, and throughput constraints.
• Implement robust error handling, monitoring, and recovery mechanisms.

Python & Custom Development

• Use Python for transformations, scripting, automation, and custom processor logic (where applicable).

Governance, Security & Operations

• Apply best practices for secure data movement, authentication, authorization, and encryption.
• Assist with flow versioning, deployment processes, and operational runbooks.
• Collaborate with internal teams to ensure platform changes align with operational and compliance requirements.

Advisory & Collaboration

• Act as a technical advisor to internal data and platform teams.
• Clearly document findings, recommendations, and implemented improvements.
• Provide knowledge transfer and guidance to ensure long-term sustainability.

Required Experience & Skills

• Strong knowledge of ETL/ELT concepts and modern data pipeline architectures.
• Proficiency in Python for data processing, scripting, and automation.
• Experience working with Linux-based systems and distributed environments.
• Ability to analyze existing implementations and deliver actionable optimization recommendations.

Preferred Experience

• Familiarity with CI/CD practices for data pipelines.
• Experience with monitoring and tuning distributed data platforms.
• Exposure to cloud and hybrid data architectures.

Engagement Expectations

• Deliverables-focused, hands-on consulting role.
• Ability to work independently while collaborating with internal stakeholders.
• Clear written and verbal communication of technical findings and recommendations.