Data Engineer

*****************************Immediate Joiners Only************************************

Creospan is hiring for Azure Databricks Engineer for one of our Banking & Wealth Management projects based in Pune, India.

Experience: 8+ Years

Location: Pune (Hybrid)

About the Role:

We are looking for an experienced Azure Databricks Engineer to join a high-impact data engineering team working on enterprise-scale Azure data platforms. The role involves designing, building, and optimizing scalable data pipelines using Azure Databricks and Apache Spark. You will collaborate with global stakeholders to deliver secure, high-performance analytics and data processing solutions.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using Azure Databricks
• Build and optimize batch and streaming ETL/ELT workflows using Apache Spark
• Develop data processing solutions using PySpark, Spark SQL, and Delta Lake
• Integrate data from Azure sources such as ADLS Gen2, Azure SQL, APIs
• Optimize Spark jobs for performance, cost efficiency, and reliability
• Implement data quality, validation, and reconciliation frameworks
• Collaborate with data architects, analysts, and business stakeholders
• Ensure compliance with data security, governance, and regulatory standards
• Support production deployments and troubleshoot pipeline failures

Required Skills & Qualifications:

• 8+ years of overall Data Engineering experience
• Hands-on Azure Databricks experience
• Strong expertise in Apache Spark, PySpark, Spark SQL
• Experience with Azure Data Lake (ADLS Gen2)
• Hands-on experience with Azure Data Factory (ADF) for orchestration
• Strong SQL skills and experience with relational databases
• Python proficiency for data processing and automation
• Experience with CI/CD pipelines (Azure DevOps preferred)
• Working knowledge of Azure cloud security and access controls

Company Profile:

Creospan is a subsidiary of Creospan Inc., headquartered in Chicago, IL. Since 1999, we have grown into a leading technology consulting and solutions provider, serving global clients across Banking, Telecom, Technology, Manufacturing, Ecommerce, Insurance, Transportation, and Healthcare industries. Our team specializes in emerging technologies, helping clients build next-generation products that drive innovation.

🌐 For more details, visit: www.creospan.com