Technical Lead-Data Engg

Noida, Uttar Pradesh, India

2 months ago

Applicants: 0

Salary Not Disclosed

3 days left to apply

Job Description

Area(s) of responsibility JD For Azure Data Bricks Core responsibilities Design And Development Architect and build scalable, robust data solutions on the Azure Databricks platform. Data Pipelines Develop and maintain data pipelines, handling both batch and streaming data for various file formats like CSV, JSON, and Parquet. ETL/ELT Processes Implement and optimize ETL/ELT processes to transform raw data into usable formats for analytics and reporting. Collaboration Work closely with data engineers, data scientists, and business analysts to meet data requirements and support machine learning workflows. Performance Optimization Tune Databricks clusters and jobs for performance, scalability, and cost efficiency. Integration Integrate Databricks with other Azure services such as Azure Data Lake, Synapse, Key Vault, and SQL Database. Maintenance And Support Monitor and troubleshoot data pipelines, resolve issues, and perform code reviews. Security And Governance Implement security best practices, access controls, and data governance policies. Technical Proficiency Required skills and experience Strong expertise in Azure Databricks and its components. Proficiency in programming languages like PySpark and Python. Strong SQL skills. Azure services: Experience with other Azure services like Azure Data Factory, Azure Data Lake Storage (ADLS), and Azure SQL Database