Lead I - Airflow Migration Developer

Job Description

Role Description Job Summary: We are seeking an experienced Airflow Migration Developer to lead the migration of job scheduling processes from Autosys to Apache Airflow . The ideal candidate will have a strong background in batch processing, workflow orchestration, and cloud deployment, particularly on AWS . This role involves close collaboration with application, data, and cloud engineering teams to ensure a seamless transition, robust scheduling, and high availability in production environments. Key Responsibilities Analyze existing Autosys job definitions, dependencies, and workflows. Design and implement equivalent Apache Airflow DAGs that maintain or improve scheduling efficiency and SLAs. Integrate Airflow with MarkitEDM and other downstream systems. Ensure proper logging, monitoring, and ing aligned with SLA and operational standards. Collaborate with cloud engineering and application teams to deploy workflows in AWS environments (e.g., ECS, Lambda). Create detailed documentation for migration logic, configurations, and operational runbooks. Support User Acceptance Testing (UAT) and manage production rollout activities. Mandatory Skills Strong hands-on experience with Apache Airflow, including DAG design and scheduling. Proficient with Autosys job scheduling and batch job management. Proficiency in Python for scripting and workflow orchestration. Solid experience with AWS services: ECS, S3, Lambda, CloudWatch. Familiarity with MarkitEDM or similar data management platforms. Knowledge of CI/CD pipelines and DevOps practices. Strong debugging, problem-solving, and performance tuning skills. Excellent communication and documentation capabilities. Good-to-Have Skills Experience in financial services or enterprise-scale data environments. Knowledge of Airflow deployment in containerized environments (e.g., Docker, Kubernetes). Awareness of audit, compliance, and security requirements in batch processing. Familiarity with infrastructure automation and Infrastructure as Code (IaC) tools. Skills Apache Airflow,Autosys job scheduling ,Python,Markit Edm