Data Engineer (AWS)

Job Description

Skills: ETL + Python + PySpark + SQL + Cloud (AWS) Technologies: Python, AWS CloudWatch, AWS IAM, Amazon S3, AWS DevOps, AWS Glue Catalog, AWS Glue ETL, Apache Spark, AWS Glue Studio, Spark Optimization We are seeking a Sr Developer with 5-8 years of experience to join our dynamic team. Ideal candidate will have strong expertise in Data Engineering: ETL + Python + Pyspark + SQL + Cloud (AWS). This role focuses on developing and optimizing data solutions using AWS services. Candidates contribute to improving data efficiency and innovation in processing capabilities. Develop and optimize data processing solutions using Apache Spark to enhance performance and scalability. Implement AWS CloudWatch for monitoring and logging to ensure system reliability and performance. Utilize AWS DevOps practices to streamline deployment processes and improve operational efficiency. Manage AWS IAM roles and policies to ensure secure access and compliance with organizational standards. Design and implement data workflows using AWS Glue Studio to automate ETL processes. Leverage AWS Glue ETL and AWS Glue Catalog to manage and transform data effectively. Store and manage data efficiently using Amazon S3. Ensure data quality and integrity throughout the continuous improvement lifecycle. Conduct performance tuning and Spark optimization for enhanced resource utilization. Provide technical expertise in integration of data engineering systems and processes. Collaborate with cross-functional teams to integrate data solutions into business processes. Stay updated with the latest AWS technologies and practices relevant to data development. Exhibit expertise in Python programming for developing reliable data applications. Additional Requirements: Strong experience in Spark optimization and AWS CloudWatch. Proficiency in AWS DevOps and AWS IAM for secure and efficient operations. Hands-on experience with AWS Glue Studio, AWS Glue ETL, and AWS Glue Catalog. Strong Python expertise.