Hadoop Administrator

We are seeking an experienced Hadoop Administrator to manage, maintain, and optimize our big data infrastructure. The ideal candidate will be responsible for the deployment, configuration, monitoring, and performance tuning of Hadoop clusters in a production environment.

Key Responsibilities:

• Install, configure, and maintain Hadoop clusters (HDFS, YARN, MapReduce) across development, test, and production environments.
• Manage cluster capacity planning, scaling, commissioning, and decommissioning of nodes.
• Monitor cluster health, performance, and resource utilization using tools such as Ambari, Cloudera Manager, or Ganglia/Nagios.
• Implement and manage security via Kerberos, Ranger, Knox, and LDAP integration.
• Perform backup, recovery, and disaster-recovery planning for HDFS data.
• Troubleshoot and resolve cluster-related issues, job failures, and performance bottlenecks.
• Administer ecosystem components: Hive, HBase, Spark, Sqoop, Oozie, Kafka, and Zookeeper.
• Apply patches, upgrades, and routine maintenance with minimal downtime.
• Automate routine administrative tasks using shell scripting, Python, or Ansible.
• Collaborate with data engineering and DevOps teams to support data pipelines.

Required Skills:

• Hands-on experience administering Hadoop distributions (Cloudera CDH/CDP, Hortonworks HDP, or Apache Hadoop).
• Strong knowledge of HDFS, YARN, and the broader Hadoop ecosystem.
• Proficiency in Linux/Unix administration (RHEL, CentOS, or Ubuntu).
• Experience with Kerberos and cluster security frameworks.
• Scripting skills in Shell, Python, or Bash.
• Familiarity with monitoring and automation tools.