Lead AI Infrastructure & Distributed Systems Engineer

Job Title : Lead AI Infrastructure & Distributed Systems Engineer

Experience: 3–5 Years

Location: Mumbai

Notice Period: Immediate Joiners Preferred

Employment Type: Full-Time

We are seeking a Lead AI Infrastructure & Distributed Systems Engineer to design and optimize high-performance GPU infrastructure for training large-scale AI models. This role focuses on building efficient multi-node GPU clusters, optimizing distributed training, and maximizing hardware performance for enterprise AI workload

Key Responsibilites

• Design and build distributed GPU training infrastructure for large language models (LLMs)
• Implement distributed training using PyTorch FSDP, DeepSpeed (ZeRO-3), and Megatron-LM.
• Configure Pipeline Parallelism (PP) and Tensor Parallelism (TP) across multi-node GPU clusters.
• Optimize training performance through gradient checkpointing, CPU RAM offloading, and NCCL communication tuning.
• Improve Linux system performance, PCIe bandwidth utilization, and networking efficiency (10GbE/100GbE).
• Monitor and optimize GPU cluster health, thermal efficiency, and resource utilization.
• Troubleshoot distributed training bottlenecks and improve overall infrastructure scalability.
• Collaborate with AI researchers and ML engineers to support large-scale model training.

Required Skills

• 3+ years of experience in AI infrastructure or distributed machine learning systems.
• Strong hands-on experience with PyTorch FSDP, DeepSpeed, Horovod, or similar distributed training frameworks.
• Expertise in Python and familiarity with C/C++ for system-level configuration.
• Strong knowledge of Linux administration, system tuning, and performance optimization.
• Experience with NCCL, CUDA, PCIe Gen4/Gen5, GPU communication, and multi-node training.
• Understanding of transformer architectures and distributed LLM training.
• Experience managing GPU clusters and high-performance computing (HPC) environments.
• Excellent analytical, troubleshooting, and problem-solving skills.

Preferred Skills

• Experience with Megatron-LM or other large-scale LLM training frameworks.
• Knowledge of Kubernetes, Docker, Slurm, or cluster orchestration tools.
• Experience with InfiniBand or high-speed networking technologies.
• Familiarity with NVIDIA profiling and performance optimization tools.