Senior Data Engineer – GCP | BigQuery | Dataflow | Dataproc

Additional Important Note For Applicants

• Currently, only immediate joiners (who have already completed their notice period) or candidates serving a notice period of up to 30 days will be considered for this opportunity.
• Candidates with longer notice periods may not be considered at this stage due to urgent project requirements.
  
  

Important Note for Applicants

Kindly read the job description carefully before applying. Please apply only if your experience, technical skills, and notice period align with the mandatory requirements mentioned above. Profiles that do not meet the core criteria may face rejection during the screening process, which can lead to unnecessary time and effort from both sides. We appreciate your understanding and cooperation.

Senior Data Engineer – GCP | BigQuery | Dataflow | Dataproc

Location: Chennai, India

Experience Required: 7–11 Years

Job Openings: 2 Positions

Work Mode: Hybrid

Shift Type: Day Shift

Employment Type: Full-time,30088

Role Overview

We are looking for an experienced Senior Data Engineer with strong expertise in Google Cloud Platform (GCP), big data technologies, data pipeline development, and cloud-based data architecture.

The ideal candidate will be responsible for designing, developing, and maintaining scalable data platforms, data pipelines, and processing solutions. The role involves working with large-scale data systems, cloud technologies, automation frameworks, and collaborating with cross-functional teams to deliver reliable and efficient data solutions.

Key Responsibilities

• Design, build, and maintain scalable data engineering solutions including data pipelines, data platforms, and data processing frameworks.
• Develop and optimize data solutions using Google Cloud Platform services.
• Design and implement data architectures including data warehouses, data lakes, and lakehouse solutions.
• Build and maintain ETL/ELT pipelines for structured and unstructured data processing.
• Develop data transformation workflows using PySpark and Python.
• Design and implement real-time and batch data processing solutions.
• Work with cloud data services including BigQuery, Dataflow, Dataproc, Data Fusion, and Cloud SQL.
• Develop and manage workflow orchestration using Apache Airflow.
• Implement infrastructure automation using Terraform and CI/CD practices using Tekton.
• Build and optimize data models to support analytics, machine learning, and business intelligence requirements.
• Monitor, troubleshoot, and optimize data systems for performance, scalability, and reliability.
• Implement data governance, security, and compliance best practices.
• Develop APIs and integrate data solutions with enterprise applications.
• Collaborate with data scientists, analysts, architects, and business stakeholders.
• Mentor junior team members and contribute to engineering best practices.
  
  

Must-Have Skills

• 7+ years of experience in Data Engineering.
• Strong hands-on experience with Google Cloud Platform (GCP).
• Expertise in the following GCP services:
• BigQuery
• Dataflow
• Dataproc
• Data Fusion
• Cloud SQL
• Strong experience with Terraform and infrastructure-as-code practices.
• Hands-on experience with Tekton and CI/CD pipeline implementation.
• Strong knowledge of Apache Airflow workflow orchestration.
• Experience working with Airflow and PySpark-based data processing workflows.
• Proficiency in Python for data engineering and automation.
• Strong experience with PostgreSQL and relational database concepts.
• Experience with data pipeline development, optimization, and troubleshooting.
• Experience working with large-scale data processing systems.
• Good understanding of APIs, data integration, and streaming concepts.
• Strong analytical and problem-solving skills.
  

Nice-to-Have Skills

• Experience with Kafka and Google Pub/Sub for real-time data streaming.
• Experience with Jira or similar project management tools.
• Knowledge of data governance frameworks and best practices.
• Experience with machine learning platforms and model deployment workflows.
• Familiarity with data visualization and business intelligence tools.
• Experience with cloud data architecture and multi-cloud environments.
• Knowledge of containerization technologies and Kubernetes.
• Experience implementing monitoring and reliability practices for data platforms.
• Understanding of security and compliance practices in cloud environments.
  
  

Required Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or related field.
• Strong experience designing and implementing enterprise-scale data solutions.
• Ability to work independently and collaborate effectively with cross-functional teams.
• Excellent communication skills with the ability to explain complex technical concepts.
  
  

Interview Process

• 2 rounds of interviews.
• Interview Mode: Video/Virtual.
• Candidates should be available for further discussions within a short turnaround time.
  
  

Candidate Preference

• Candidates available with a short notice period are preferred.
• Candidates with stable work experience and strong hands-on GCP Data Engineering expertise are preferred.
  
  

Skills: terraform,data fusion,data engineering,python,gcp,infrastructure as code,ci/cd,cloud sql,apache airflow,tekton,airflow + pyspark,dataproc,bigquery,dataflow,postgresql