Python Pyspark SDET

Job Description

Job Title: SDET ? PySpark/Python Data Engineering (Azure Cloud) Location: Indore | Pune | Bangalore | Chennai Experience Required: 5+ years Role Overview: Seeking a skilled Software Development Engineer in Test (SDET) with expertise in PySpark and Python for data engineering and data pipeline validation on Azure Cloud. You will play a critical role in building automation frameworks, developing and executing test cases to ensure the quality and reliability of large-scale data solutions. Key Responsibilities: Develop, maintain, and execute automated test scripts for data pipelines and engineering workflows using Python and PySpark. Validate data transformations, data quality, and pipeline performance on Azure Cloud environments (preferably using Databricks). Collaborate with data engineers and developers to understand data workflows and test scenarios. Build and optimize test frameworks that integrate with CI/CD pipelines. Perform thorough regression, performance, and API testing for data applications. Report and track defects, and work closely with engineering teams for resolution. Skills & Qualifications: Strong programming skills in Python and PySpark. Experience in automated testing for data pipelines and big data platforms. Working knowledge of Azure Cloud services, including Azure Data Factory and Databricks. Familiarity with CI/CD tools and test automation frameworks. Understanding of data validation, ETL testing, and SQL query optimization. Excellent problem-solving and communication skills. Preferred: Hands-on experience with Databricks and related Azure data technologies. Background in software development and test engineering for data-intensive applications.