Job Description

Job Description Job Title: Data Engineer (PySpark + Azure Databricks) Location: Bangalore, Hyderabad, Chennai & Mumbai Experience: 6 ? 10 years Job Summary We are looking for a skilled Data Engineer with strong experience in PySpark and Azure Databricks to design, build, and optimize scalable data pipelines and analytics solutions. The ideal candidate will have hands-on expertise in big data processing, Azure data services, and automation for data-driven decision-making. Key Responsibilities Design, develop, and maintain ETL/ELT data pipelines using Azure Data Factory and Databricks. Build and optimize PySpark scripts and Databricks notebooks for large-scale data processing. Perform data ingestion, transformation, and aggregation from multiple sources (SQL, APIs, ADLS, Blob, SFTP, etc.). Implement Delta Lake architecture for data versioning, ACID transactions, and optimized queries. Work with Azure Data Lake Storage (Gen2) for managing and structuring raw, processed, and curated data layers. Develop and manage CI/CD pipelines for Databricks using Azure DevOps or GitHub Actions. Implement data validation, quality checks, and monitoring mechanisms across data pipelines. Collaborate with business and analytics teams to deliver clean, structured data for reporting and machine learning. Optimize PySpark jobs using techniques like broadcast joins, Z-Ordering, partitioning, and caching. Ensure compliance with data governance, security, and regulatory requirements. Troubleshoot performance bottlenecks and improve resource utilization on Databricks clusters. Skills Required RoleData Engineer (Pyspark + DataBricks) Contract Hiring Industry TypeIT/ Computers - Software Functional Area Required Education Bachelor Degree Employment TypeFull Time, Permanent Key Skills DATA ENGINEER DATABRICKS PY SPARK Other Information Job CodeGO/JC/1327/2025 Recruiter NameChristopher