Data Engineering Lead

Role Overview

Tata Consulting Engineers (TCE) is looking for a highly experienced Data Engineering Lead to drive the organization's enterprise-wide data transformation journey. This role will be responsible for designing, implementing, and operating TCE's Single Enterprise Data Platform, building a modern Data Lake / Lakehouse architecture, and enabling the organization's AI, Analytics, Digital Twin, and Agentic AI ambitions.

The candidate will serve as both an internal enterprise platform owner and a customer-facing data consulting leader, helping industrial clients architect and implement modern data platforms aligned with Industry 4.0, Digital Twin, and AI initiatives.

This is a hands-on leadership role requiring deep expertise in Databricks, SAP integration, enterprise data architecture, governance, security, and cloud-based data engineering.

Educational Qualification (Engineering degree is mandatory)

• Bachelor's degree in Engineering, Computer Science, Information Technology, or related field.
• Master's degree preferred.

Key Responsibilities

Enterprise Data Platform Ownership

• Define and implement TCE's enterprise data strategy and roadmap.
• Build and manage TCE's centralized Data Lake/Lakehouse platform.
• Establish a single source of truth for structured and unstructured enterprise data.
• Develop scalable data architectures supporting AI, ML, Generative AI, Digital Twin, BI, and advanced analytics.
• Drive enterprise-wide data democratization while ensuring security and governance.
• Create reusable enterprise data products and domain-specific data models.

Data Engineering & Integration

• Design and implement large-scale data ingestion, transformation, and orchestration pipelines.
• Integrate enterprise applications including:
• SAP ECC / S4 HANA
• SAP BW
• ERP systems
• PLM Systems
• CRM Applications
• HRMS Platforms
• Engineering applications
• Project Management Systems
• Document Management Systems
• Build real-time and batch data pipelines.
• Implement Delta Lake architecture and Lakehouse principles using Databricks.

AI & Advanced Analytics Enablement

• Create AI-ready data foundations for:
• Generative AI
• Agentic AI
• Knowledge Graphs
• Digital Twins
• Advanced Analytics
• Implement semantic layers, metadata management, and data catalogs.
• Partner with AI and Digital Twin teams to enable enterprise-scale AI solutions.

Data Governance, Security & Compliance

• Define enterprise data governance frameworks.
• Establish data quality, lineage, metadata, and master data management practices.
• Implement RBAC, ABAC, data masking, encryption, and privacy controls.
• Ensure compliance with enterprise security policies and regulatory requirements.
• Build governance processes for data ownership, stewardship, and lifecycle management.

Customer-Facing Industrial Data Platform Engagements

• Lead client workshops and discovery sessions.
• Architect industrial data platforms for energy, manufacturing, metals, infrastructure, transportation, and process industry clients.
• Support solutioning and proposal development for Data & AI opportunities.
• Work closely with strategic partners such as:
• Databricks
• Microsoft Azure
• Cognite
• NVIDIA
• Provide technical leadership during client implementations.

Leadership & Team Building

• Build and mentor a high-performing data engineering team.
• Drive engineering best practices and DevSecOps culture.
• Establish standards for architecture, coding, testing, deployment, and operations.
• Review solution designs and ensure platform scalability and maintainability.

Required Technical Skills

Mandatory

• 10+ years of Data Engineering experience.
• 5+ years of hands-on Databricks implementation experience.
• Proven experience in building enterprise Data Lakes/Lakehouses.
• Strong expertise in:
• Databricks
• Delta Lake
• Apache Spark
• PySpark
• SQL
• Python
• Experience integrating SAP systems:
• SAP ECC
• SAP S/4HANA
• SAP BW
• SAP Datasphere
• SAP Data Services
• Expertise in ETL / ELT design and implementation.

Cloud Platforms

Strong experience in at least one:

• Microsoft Azure (Preferred)
• AWS
• Google Cloud

Preferably hands-on experience with:

• Azure Data Factory
• Azure Synapse
• Azure Fabric
• Azure Data Lake Storage
• Event Hub
• Key Vault

Governance & Security

Hands-on experience with:

• Unity Catalog
• Data Governance Platforms
• Data Lineage
• Data Catalogs
• Data Quality Frameworks
• MDM Solutions
• Enterprise Security Architecture

Integration Experience

Experience integrating:

• SAP
• Oracle ERP
• Salesforce
• ServiceNow
• SharePoint
• Document repositories
• Engineering applications
• Industrial OT systems

Preferred Domain Experience

Candidates with experience in one or more of the following industries will be preferred:

• Engineering & EPC
• Manufacturing
• Metals & Mining
• Energy & Utilities
• Infrastructure
• Transportation & Rail
• Oil & Gas
• Smart Cities
• Industrial Digital Transformation