Jobs at Absa

A Hadoop Engineer is a professional responsible for programming Hadoop applications and knows about all the components or pieces of the Hadoop Ecosystem , understands how the Hadoop components fit together and has the ability to decide on which is the best Hadoop component for a specific task.

The Data Operations team within CTO, is looking for a highly talented Hadoop Engineer to join their team in Johannesburg.
The Data Operations team is responsible for supporting all the Applications on the Hadoop ecosystem. This role expands in maintaining changes on datasets and optimisation activities on all Applications, including new development.
They therefore need to understand basic programming to enable them to manage Big Data and to transfer all data to Hadoop.

Job Description
Play a lead role in the Data Operations Team

• Prioritise tasks assign to the team.
• Give input and quality assure the tasks allocated to the team.
• Give input and feedback to team members on progress of tasks.
• Identify optimisation opportunities.
• Improvement on recovery time in case of prod failures.
• Test prototypes and oversee handover to the operational teams.
• Attend and contribute to regular project meetings.
• Responsible for the actual coding or programming of Hadoop applications.
• Propose best practices and standards.
• Maintain security and data privacy.
• High-speed querying.
• Translate complex functional and technical requirements into detailed design.

Job Experience & Skills Required:

• 5 years' relevant Hadoop experience
• Min one year experience with Scala programming language (advantageous)
• Familiarity with Hadoop ecosystem and its components.
• Good knowledge of the concepts of Hadoop.
• Solid experience in a working environment in Big Data development utilising SQL or Python.
• Experience in Big Data development using Spark.
• Experience in Hadoop, HDFS and MapReduce.
• Experience in database design, development and data modelling.
• Data architecture and security.

The following additional knowledge, skills and attributes are preferred:

• Good knowledge in back-end programming, specifically java.
• Experience with development in a Linux environment and its basic commands.
• Ability to write reliable, manageable, and high-performance code.
• Should have basic knowledge of SQL, database structures, principles, and theories.
• Knowledge of workflow/schedulers.
• Strong collaboration and communication skills.
• Strong analytical and problem solving skills.