Hadoop System Administrator at Moyo Business Advisory

Job description

Responsibilities include:

• Administer and maintain Hadoop clusters across all environments – Production, UAT, Development and DR.
• Installation/upgrade/configuration of the Hadoop platform.
• Proactively monitor cluster health and perform performance tuning activities.
• Perform capacity planning and expansion activities working across Infrastructure and other Enterprise Services teams
• Perform cluster maintenance with patching, backup/recovery, user provisioning, automation of routine tasks, troubleshooting of failed jobs, configure and maintain security policies.

Qualifications

• ±5 years of related IT experience
• Bachelor's Degree in Information Technology, Engineering, Computer Science, related field
• 2+ years of hands on experience with enterprise level Hadoop administration
• Strong understanding of one or more major RDBMS technologies like, MS-SQL, Oracle, DB2, etc.
• Understanding of High Availability, Disaster Recovery and Storage technologies like Data Replication, SRDF/non-SRDF
• Strong Linux/Unix and Networking experience

Preferred Skills

• In depth understanding of Hadoop ecosystem, such as
• Hive, Pig, Spark, Sqoop, Kafka
• Impala, Flume, Mahout, H2O
• Experience with No-SQL database platforms like MongoDB will be a big plus

Experience in:

• Installation/configuration Hadoop clusters from scratch.
• Upgrading Hadoop clusters using rolling upgrade/express upgrade methods.
• troubleshooting and analyzing Hadoop cluster services/component failures and job failures.
• Setup, configuration and management of security for Hadoop clusters at an Enterprise level.
• Setting up Ranger policies for HDFS and Hive.
• Implementation experience in setting and configuring
• Backup and recovery procedure on Hadoop.
• High availability and Disaster Recovery procedures.

Experience in:

• Managing Hadoop cluster and developing custom tools/scripts to monitor the Hadoop Cluster health.
• Configuration and tuning of various components like Hive, Spark, Pig.
• Documenting Runbooks, and other operational documentation.