Data Engineer II, Amazon Now

Job Description

Description We are SCOPE(Supply Chain Operations, Planning and Efficiency) team, a part of Amazon Now(Tez). We are innovative quick-commerce offering that delivers everyday essential products to customers in just 10 minutes. We build systems to peer into the future and estimate the most cost-effective way to distribute tens of millions of products every week to Amazon warehouses. Our team utilizes the latest applications in science, machine learning, and scalable distributed software on the Cloud to automate and optimize inventory and shipments under the ever-changing landscape of demand, pricing and supply. We are data-driven, dive deep, and make decisions based on data while proactively managing risks and seeing the bigger picture. The team strives for simplistic and intuitive solutions that reduce complexity, add transparency, and improve visibility. We're seeking a Data Engineer II who will own the near real-time data infrastructure powering our AI/ML based forecasting platform for SCOPE. This role focuses on building high-performance streaming pipelines, optimizing embedding freshness, and implementing global latency strategies to ensure we deliver up-to-date, low-latency insights across QC network. You will play critical role in scaling AI-driven analytics across multiple regions while balancing performance and cost. Key job responsibilities Design and implement streaming data pipelines to process high-volume, near real-time data from multiple sources. Build and maintain the infrastructure supporting large language models, including embedding generation, vector storage, and retrieval systems. Develop and optimize a modern data lakehouse to support both batch and real-time analytics workloads. Implement caching strategies, query optimization, and multi-region deployment to achieve sub-second response times. Balance performance requirements with cost considerations through efficient resource utilization and workload optimization. Ensure data reliability, freshness, and compliance across the entire data pipeline. Basic Qualifications 3+ years of data engineering experience 4+ years of SQL experience Experience with data modeling, warehousing and building ETL pipelines Experience with AWS services including S3, Redshift, Sagemaker, EMR, Kinesis, Lambda, and EC2 5+ years of SQL experience Preferred Qualifications Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases) Knowledge of cloud computing services or deployment architecture Experience building data pipelines or automated ETL processes Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you?re applying in isn?t listed, please contact your Recruiting Partner. Company - ASSPL - Karnataka Job ID: A3114917