Lead II - Data Engineering-AWS

Role Description

Job Description

UST is looking for a skilled AWS Data Lead to join team with 8 to 10 years of experience and contribute to the development and maintenance of robust data pipelines and systems on the AWS platform, with a focus on leveraging PySpark for efficient data processing. As a Data Engineer, role will play a crucial role in ensuring the efficient flow, storage, and processing of data for our organization.

Responsibilities

Data Pipeline Development: Design, implement, and maintain scalable and efficient data pipelines on the AWS platform using tools such as AWS Glue, Apache Spark, and PySpark.

AWS Services Utilization: Leverage AWS services like S3, Glue, Athena, and others to build end-to-end data solutions.

ETL Processing with PySpark: Develop and optimize ETL processes using PySpark to facilitate seamless data extraction, transformation, and loading.

Requirements

Experience: Proven experience as a Data Lead with a focus on AWS data solutions, including hands-on experience with PySpark.

Programming Skills: Proficient in Python, with advanced experience in PySpark for efficient data processing.

ETL Tools: Hands-on experience with AWS Glue or other ETL tools for data processing.

Skills

data engineering,aws,aws glue,pyspark,etl,