Azure Data Engineer

Job Description

Job Role: Azure Data Engineer Experience: 7+ Years Location: Hyderabad Job Description: We are looking for a talented and driven Data Engineer to join our expanding team. In this role, you will design, build, migrate, and maintain data solutions on the Azure platform. You?ll play a key role in modernizing our legacy data systems, developing high-performance data pipelines, and delivering scalable data warehousing architectures. The ideal candidate is a hands-on expert in Azure data services, with strong proficiency in T-SQL, Azure Data Factory (ADF), Power BI and Python, and a deep commitment to data quality, reliability, and system performance. Roles & Responsibilities: Data Pipeline Development: Develop, orchestrate, and automate end-to-end ETL/ELT pipelines using Azure Data Factory (ADF), complete with CI/CD integration for robust deployment. Data Warehousing: Design, optimize, and manage large-scale data warehousing solutions in Azure SQL Data Warehouse, including tuning stored procedures for parallel execution. Legacy System Data Migration: Lead the migration of legacy systems (e.g., SSIS, on-prem SQL) to the Azure cloud, ensuring zero data loss and maintaining data consistency throughout the process. SQL Development & Tuning: Write complex T-SQL queries, stored procedures, and functions; conduct performance tuning on SQL jobs to ensure efficiency. Data Lake Architecture: Design and manage a multi-layered data lake architecture (raw, curated, processed) using Azure Data Lake Storage (ADLS). Data Modeling: Design and manage both relational and schema-less (e.g., MongoDB) data models and integrate them with Azure-based applications. Business Intelligence: Develop interactive Power BI dashboards, semantic models, and complex DAX calculations, ensuring real-time data refresh and adherence to governance standards. Performance Optimization: Implement caching layers (e.g., Redis) to optimize performance in data-intensive applications. Data Quality & Governance: Implement and manage validation frameworks using T-SQL and custom Python scripts to ensure data quality, consistency, and security.