Data Engineer - Rosebank Towers at Absa Group Limited (Absa)

Job Summary

• As a Data Engineer , you will be responsible for ensuring that the specific data requirements of the business units are met on Hadoop. You will work closely with the Data Analysts, Data Scientists, and stakeholders to understand the data needs and develop and maintain the data pipeline and infrastructure to support those needs. You will be responsible for designing, building, and maintaining data systems and applications on Hadoop that meet the business unit's requirements.

Job Description

Responsibilities:

• Work closely with the Data Analysts, Data Scientists, and stakeholders to understand the business unit's data requirements.
• Design, build, and maintain data pipelines and data systems on Hadoop to support the specific data needs of the business unit.
• Develop data models and schemas that support the business unit's specific requirements on Hadoop.
• Develop and maintain data quality controls and data governance processes to ensure data accuracy, completeness, and consistency on Hadoop.
• Develop and maintain ETL processes to integrate data from various sources on Hadoop.
• Work with the IT team to ensure that data systems and applications on Hadoop are integrated and scalable.
• Develop and maintain data documentation and knowledge base for end-users and stakeholders.
• Collaborate with other Data Engineers, Data Analysts, and Data Scientists to ensure the smooth functioning of the data pipeline and infrastructure on Hadoop.
• Identify, diagnose, and report data quality issues to Data Analysts, Data Scientists, and Data Engineers.
• Stay current with industry trends, best practices, and emerging technologies in data management and analytics on Hadoop.
• Continuously improve data-related processes, systems, and tools to enhance the efficiency and effectiveness of the data team on Hadoop

Qualifications:

• Bachelor's degree in Computer Science, Information Systems, or a related field.
• 3+ years of experience in data engineering or related field.
• Strong proficiency in SQL and one or more programming languages (Python, Java, Scala, etc.).
• Experience in data modeling and data architecture on Hadoop.
• Experience in developing and maintaining ETL processes on Hadoop.
• Familiarity with Hadoop ecosystem and tools (HDFS, Hive, HBase, Sqoop, Spark, etc.).
• Strong problem-solving and analytical skills.
• Excellent communication and interpersonal skills.
• Ability to work collaboratively in a team-oriented environment.
• Strong attention to detail and ability to prioritize tasks

Education

• Bachelor`s Degrees and Advanced Diplomas: Manufacturing, Engineering and Technology