Back to Jobs

Lead Data Engineer- GenAI/LLM

Actively Reviewing

Altraize

Coimbatore Full-Time 2–4 yrs exp Posted 3 hours ago · Apply by Sep 14, 2026

Job Description

10+ years of hands-on data engineering experience, including technical leadership of data platform initiatives

Strong, hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake, Delta Live Tables, Databricks Workflows, and Unity Catalog
Proven experience designing and implementing the medallion (bronze/silver/gold) architecture for data ingestion, curation, and consumption
Strong expertise in distributed data processing using Apache Spark (PySpark and Spark SQL) or equivalent big-data frameworks
Proven experience designing and building ETL/ELT pipelines for both batch and streaming data at scale
Expert-level proficiency in Python and SQL for data transformation, validation, and pipeline development
Strong experience building and managing data lakes and data warehouses, including dimensional and lakehouse data modeling
Working knowledge of Large Language Models (LLMs) and GenAI concepts — prompts, embeddings, vector databases, and Retrieval-Augmented Generation (RAG) — and the data pipelines required to support them
Hands-on experience with at least one major cloud platform (AWS, Azure, or GCP) and its core data services
Proven experience leading large-scale data migrations from on-premise (e.g. Hadoop) and cloud platforms to modern data architectures
Strong understanding of distributed data processing, partitioning, and performance optimization techniques
Experience implementing data security, governance, lineage, and access control (e.g. IAM, encryption, cataloging)
Strong understanding of object-oriented programming, software design patterns, and CI/CD practices
Familiarity with Agile/Scrum delivery methodologies and experience mentoring engineers
Excellent problem-solving, analytical, and stakeholder-management skills
Strong verbal and written communication skills

Skills: pipelines,data,aws