Manager-Data Engineering-Cloud Data Engineering

Job Description

Job Summary Data Engineer Consultant ? Job Description Data Engineer (DE) Consultant is responsible for designing, developing, and maintaining data assets and data related products by liaising with multiple stakeholders.. Qualifications And Skills Strong knowledge on Python and Pyspark - Expectation is to have ability to write Pyspark scripts for developing data workflows. Strong knowledge on SQL, Hadoop, Hive, Azure, Databricks and Greenplum - Expectation is to write SQL to query metadata and tables from different data management system such as, Oracle, Hive, Databricks and Greenplum. Familiarity with big data technologies like Hadoop, Spark, and distributed computing frameworks. Expectation is to use Hue and run Hive SQL queries, schedule Apache Oozie jobs to automate the data workflows. Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing. - Expectation is to have strong problem-solving and troubleshooting skills. Expectation is to establish comprehensive data quality test cases, procedures and implement automated data validation processes. Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience. 5-7 years of experience in Data Engineer. Proficiency in programming languages commonly used in data engineering, such as Python, Pyspark, SQL. Experience in Azure cloud computing platform, such as developing ETL processes using Azure Data Factory, big data processing and analytics with Azure Databricks. Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.