DevOps Engineer - Cape Town at FirstRand Group

Job Description

We require someone to manage our Cloudera environment. Their responsibilities would be to ensure that our Cloudera installation and configuration is at optimal specifications. They’ll be required to fine-tune our Cloudera environment for optimal performance. They’ll also be responsible for managing our Linux Servers.

• Deploy, configure, Cloudera Dataflow (CDF) platforms with required components like Kafka, Nifi, Schema registry, Nifi Registry etc.
• Cluster expansion and management, Access Management, performance tuning, capacity planning, security, monitoring and alerting CDF clusters.
• Collaborate with data engineers to optimize and troubleshoot data flows in CDF especially on Nifi, Kafka.
• Implement and maintain security policies and access controls for CDF environments using Kerberos , Ranger.
• Creating alerts and dashboards utilizing different application, infrastructure monitoring tools such as Grafana, Splunk etc.
• Troubleshoot and resolve issues related to CDF
• Creating tickets/cases with Cloudera vendor for support and resolution for the issues.
• Perform routine maintenance tasks, including backups, updates, and patch management for both platforms.
• Collaborate with cross-functional teams to understand and address business requirements.
• Provide support to users on CDF tools and best practices.
• DevOps – Infrastructure orchestration using Manual/Scripts. Identify and implement opportunities for automation. Resolve infrastructure and applications security vulnerabilities.

Requirements:

• Experience on Cloudera CDF platforms and clusters management.
• Proven experience as a Cloudera Administrator and Dataflow Administrator.
• In-depth knowledge of Cloudera Manager, Ranger, Spark, Apache Nifi, Kafka Schema Registry, and other components.
• Strong Experience with Linux system administration and shell scripting.
• Good understanding of CDF
• Knowledge with cloud platforms such as AWS, Azure, GCP is a plus.
• Excellent problem-solving and communication skills.
• Understand Microsoft Active Directory
• Knowledge of how to use TLS/SSL to enforce secure encrypted connections.