Collegedunia - Data Engineer - SQL/Python

Bengaluru, Karnataka, India

8 hours ago

Applicants: 0

Apply Now

Design Python SQL Git Docker

Salary Not Disclosed

4 weeks left to apply

Job Description

Description Key Responsibilities : Design, develop, and maintain scalable data pipelines on GCP. Lead and execute data migration and modernization initiatives. Optimize ETL/ELT processes, ensuring data accuracy and quality. Collaborate with analytics and engineering teams on data-driven solutions. Ensure compliance with data security and governance Skills : Strong hands-on programming in Python Expertise with GCP tools : BigQuery, Cloud Storage, Cloud Functions/Run, Dataflow (Apache Beam), Pub/Sub Strong SQL and data modeling skills Experience with ETL/ELT, CI/CD pipelines, and Git Solid understanding of data warehousing and data Skills : Experience with Airflow or Cloud Composer Familiarity with Docker, Kubernetes, and Terraform Exposure to AWS or Azure Knowledge of data quality/testing Responsibilities : Design, develop, and maintain scalable and high-performance data pipelines on GCP. Lead and execute data migration and modernization projects. Build, optimize, and manage ETL/ELT processes, ensuring high data quality and accuracy. Collaborate closely with analytics, data science, and engineering teams to deliver data-driven solutions. Ensure compliance with data security, governance, and privacy standards. Monitor, troubleshoot, and enhance data workflows for performance and reliability. (ref:hirist.tech)