Senior Data Engineer

Job Description

Job Description: Senior Developer / Tech Lead ? Data Engineering Position: Senior Developer / Tech Lead (Total 20+ positions) Experience: 5?12 years Location: PAN India (Hybrid Mode) Employment Type: Full-time About the Role We are looking for a highly skilled Senior Developer / Tech Lead with strong expertise in Databricks , data engineering, and modern ETL frameworks. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines, enabling seamless data flow across the organization. Key Responsibilities 1. Databricks Development (Mandatory) Design, develop, and optimize data pipelines and notebooks within Databricks . Work with Delta Lake , Spark, and MLflow for scalable data processing. Implement best practices for cluster management, performance, and cost optimization. 2. Data Sourcing & Data Lake Management Source, extract, and ingest data from various internal and external systems. Develop and maintain Data Lake architectures (Azure Data Lake / AWS S3 / GCP equivalent). Ensure data quality, consistency, and availability across storage layers. 3. ETL & Data Pipeline Engineering Build end-to-end ETL/ELT pipelines for batch and real-time workloads. Perform data cleansing, data transformation, and normalization using ETL tools. Work on data modeling , schema design, and versioning in a lakehouse architecture. 4. Tools & Technology Hands-on experience with multiple ETL tools (e.g., Informatica, Talend, DataStage, SSIS, Matillion, ADF). Strong experience with Apache Spark , SQL, and Python/Scala-based data engineering frameworks. Exposure to cloud platforms (Azure/AWS/GCP) is preferred. Required Skills & Qualifications 5?12 years of strong experience in data engineering . Mandatory hands-on experience with Databricks . Experience in designing and developing scalable data pipelines . Deep understanding of Data Lakes , ETL concepts, and data ingestion frameworks. Proficiency in SQL and at least one programming language (Python/Scala/Java). Experience working with distributed data systems and big data technologies. Capability to lead technical discussions, mentor junior developers, and drive best practices. Good to Have Experience with CI/CD pipelines for data engineering. Knowledge of orchestration tools (Airflow, Azure Data Factory, AWS Glue workflows). Exposure to DevOps practices and automation. Familiarity with data governance, metadata management, and observability. Why Join Us? Opportunity to lead large-scale data engineering initiatives. Work with cutting-edge cloud and big data technologies. Collaborative environment with opportunities for growth and innovation.