Data Engineer at Sanlam Group

What will make you successful in this role:
The following are the key outcomes expected from this position:

• Delivering the required services as per the service catalogue of the team.
• Coordinate service requests with security and infrastructure teams when required.
• Load large, complex data sets to and make data available for data engineers.
• Source data from internal and external data sources, engaging with technical subject matter experts.
• Build the infrastructure required for optimal Extraction, Transformation, and Loading (ETL) of data from a wide variety of data sources using various ‘big data’ technologies.
• Help build the platform for a better data supply chain for advanced analytics that goes from experimentation into production.
• Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing models for greater scalability.
• Engage with a wide range of technical stakeholders including data scientists, business analysts, data engineers, and solutions architects.
• Work with data analysts and data engineers to understand the dynamic nature that is required to support the solution that needs to be built.
• Support data stewards to establish and enforce guidelines for data collection, quality improvements, integration, and processes.
   

The following areas of expertise are key to the successful delivery of the above outcomes:

• A solid background in SQL, information architecture and ETL procedures is required.
• Experience working on SAP Data Services, Sap Hana and/or Hadoop is preferred and highly advantageous.
• Previous scripting language experience e.g. Unix Shell scripting.
• Agile Development.
• Ability to comply with and manage reusable data assets under a governance framework.
• Advanced SQL skills.
• Ability to do effort estimations.
• Extensive working knowledge and experience in development methodologies.
• The ability to configure and build solutions in various ETL toolsets would be advantageous.
• Ability to build ETL solutions in a Hadoop environment.
   

The complexity and variety of the most common challenges and problems:

• Delivering in a complex analytic environment.
• Working with a high need for information security.
• The diligence required when resolving data and technical issues and interacting with other support teams.
• Demanding business community.
   

How the job will typically impact the business and where the impact will be felt.

• Information is an asset and it can be used strategically to gain a competitive edge in the market.
• Timely delivery required - information is key to business decisions
• Accurate delivery required - information is key to business decisions
• Access to information - information is key to business decisions
   

Qualifications
Qualification & Experience

• National Diploma in an Information Technology related discipline.
• Matric
   

Experience

• 5+ years' experience as a data engineer in a BI environment
• A strong data engineering background with a specific focus on staging high quality data.
• A solid background in SQL, information architecture and ETL procedures is required.
• Experience working on SAP Hana, Teradata as well as Hadoop is preferred and highly advantageous.
• Experience with object-oriented/object functional scripting languages: Python, Java, Scala or related is preferred but not essential.
• Understanding of Data warehousing principles based on Kimball and Data Vault patterns is preferred.
• Development using SQL and procedural extensions is required.
  
   

Knowledge And Skills

• Development of data warehousing blueprints.
• Data warehouse management
• Translates business needs into long-term architecture solutions
• Defines, designs and builds dimensional databases
• Object and data models