Senior GCP Data Engineer

Job Description

Senior Data Engineer - GCP Native Platform Location:?Fully Remote (Pan - India) Role Summary Seeking an experienced Senior Data Engineer to design and build scalable data pipelines on Google Cloud Platform. You will implement data solutions using BigQuery, Dataflow, Cloud Composer, and modern engineering practices while mentoring junior team members and driving technical excellence. Key Responsibilities Pipeline Development & Engineering Build and optimize batch and streaming data pipelines using?Dataflow (Apache Beam). Develop and maintain?Airflow DAGs?in?Cloud Composer?for workflow orchestration. Implement ELT processes using?Dataform?(SQLX) for transformations, testing, and documentation. Design and maintain?BigQuery?datasets following medallion architecture (Bronze/Silver/Gold). Create reusable pipeline templates and frameworks to accelerate development. Data Platform Implementation BigQuery:?Optimized schemas, partitioning strategies, clustering, materialized views, and performance tuning. Dataflow:?Build Apache Beam pipelines in?Python?and/or?Java?for complex transformations. Cloud Composer:?Manage Airflow workflows with dependency management and scheduling. Dataform:?Author SQLX transformations, tests, and documentation. Pub/Sub:?Implement event-driven ingestion and CDC patterns. Cloud Storage (GCS):?Architect data lake structures, access patterns, and lifecycle policies. Technical Delivery Partner with architects on technical design and platform standards. Conduct code reviews and enforce engineering best practices. Troubleshoot and optimize pipeline performance and cost. Implement data quality checks, monitoring, and observability. Support production deployments and incident resolution. Collaboration & Mentorship Mentor junior engineers on GCP best practices and data engineering patterns. Collaborate with analysts, data scientists, and cross-functional teams to enable data consumption. Document technical solutions and maintain knowledge-base artifacts. Participate in agile ceremonies and coordinate with onshore/offshore teams. Required Qualifications Experience 5+ years of Data Engineering experience; minimum 2 years working with?GCP. Hands-on experience with?BigQuery?at enterprise scale and production pipelines. Strong background in building production-grade batch and streaming pipelines. Development experience in?Python?and/or?Java. Proven track record of delivering complex data solutions. Core Technical Skills BigQuery (SQL optimization, scripting, stored procedures, partitioning). Apache Beam / Dataflow (batch & streaming). Airflow / Cloud Composer (DAG development). SQL and Python programming proficiency. Git, CI/CD pipelines and basic DevOps familiarity. Additional Skills Dataform or dbt experience. Infrastructure-as-Code (Terraform basics). Data modeling and schema design. API integration (REST) and system interfacing. Cloud Monitoring, logging, and data quality frameworks. Preferred / Nice-to-Have GCP Professional Data Engineer?certification. Experience with?Vertex AI?for ML pipeline integration. Cloud Functions?for serverless processing. Dataproc?/ Spark workloads. Real-time streaming architectures and low-latency processing. Familiarity with data governance tools (Atlan, Collibra, Dataplex). Experience with legacy system migrations. BI tools experience (Power BI, Looker). Containerization (Docker) and orchestration (Kubernetes). What We?re Looking For Strong problem-solving and analytical skills. Self-motivated and able to work independently in a remote environment. Excellent communication, documentation, and stakeholder management skills. Passion for data engineering, continuous learning, and mentoring others. Collaborative team player with a delivery-focused mindset.