AI Data Engineer

Job Description

Job Title: AI Data Engineer Location: Pan India Experience: 6+ Years Employment Type: Permanent Notice Period: Immediate Joiners / <15 Days Only About the Company Our client is a global digital transformation leader with a presence in over 50 countries. Specializing in Oracle Cloud, automation, data intelligence, and enterprise consulting, they help businesses modernize their systems and accelerate innovation at scale. Job Summary: We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in Python, Azure, CI/CD, deployment, and writing efficient data pipelines. You will be responsible for designing, developing, and maintaining robust data infrastructure and pipelines that support our data-driven decision-making processes. Key Responsibilities: AI Integration: Implement LLM-based workflows, RAG pipelines, and agentic orchestration using frameworks like LangChain, LangGraph, CrewAI, or similar. Data Pipeline Development: Design, develop, and maintain efficient and scalable data pipelines using Python and Azure tech stack. Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. Automation & Deployment: Containerize and deploy solutions using Docker, Kubernetes, and cloud-native architectures (Azure/AWS/GCP). Monitoring & Governance: Ensure data quality, observability, and compliance with security standards. Documentation: Create and maintain comprehensive documentation for data pipelines, processes, and solutions. Troubleshooting: Identify and resolve data-related issues and bottlenecks. Qualifications: Education: Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent in related field. Experience: 6-8 years of experience in data engineering, with a strong focus on Python, Azure, CI/CD, and deployment. Technical Skills: Machine Learning Fundamentals: Model training, feature engineering. Generative AI & LLMs: Fine-tuning GPT/BERT, prompt engineering, RAG pipelines. Data Engineering Expertise: Advanced ETL/ELT, handling structured/unstructured data. Cloud & Big Data Tools: Spark, Hadoop, Kafka, AWS/Azure/GCP. Programming: Python (dominant for AI), SQL, ML libraries (TensorFlow, PyTorch). Proficiency in Python, Api?s for data processing and automation. Extensive experience with Azure cloud services (Azure Data Factory, Azure Databricks, Azure SQL Database, etc.). Understanding of CI/CD principles and tools (e.g., Jenkins, GitLab CI/CD). Understandin with containerization and orchestration tools (e.g., Docker, Kubernetes). Knowledge of SQL and NoSQL databases. Familiarity with data warehousing concepts and technologies. Soft Skills: Excellent problem-solving and analytical skills. Strong communication and collaboration abilities. Ability to work independently and as part of a team. Detail-oriented with a focus on quality and accuracy. Preferred Qualifications: Experience with big data technologies (e.g., Hadoop, Spark). Knowledge of machine learning and data science concepts. Certification in Azure or other relevant technologies. Benefits: Competitive salary and performance-based bonuses. Health, dental, and vision insurance. Retirement savings plan with company match. Professional development opportunities. Flexible working hours and remote work options.