Senior Data Engineer at Hollard Insurance

Job Advert Summary

This person will design, development and maintain appropriate processes to extract the appropriate data from mainly off-platform data sources at the appropriate frequency and using the appropriate technologies and rules to enable access to data for reporting and data analysis.

This person will also assist with the development of short, medium and long-term data management strategies, paying due consideration to the HLS data strategy. Key considerations in this regard are the implementation of the off-Platform Operational Data Store, Setup of Cloud technologies and the HLS Data Warehouse/Lake. 

Key Responsibilities:  

Project/Work Planning:

• Collaborate with business users, data stewards, report developers and data analysts to integrate and expose required data.
• Ensure high quality development work through the application of standards, guidelines, quality assurance reviews and testing, paying due consideration to Hollard Group standards.
• Assist with the development of project estimates, plans, and acceptance testing guidelines for approved projects.
• Ensure that only quality solutions are deployed to production.

ETL Development:

• Design/develop/test/maintain ETL processes, programs and scripts that reflect best practices for data management, error handling, operations logging and alerting, scheduling and job control, etc.
• Maintain the overall ETL architecture by which systems are interconnected and data is passed among systems.
• Conduct root cause analysis and resolve production problems and data integrity issues and eliminate such through the application of process/system enhancements.
• Raise critical technical issues to vendors or key stakeholders (E.g. DBA’s) in a timely manner where they would impact on delivery and quality of services and follow up on the technical resolution of such issues.
• Validate the data integrity and develop and maintain data validation routines.

Data Modelling:

• Understand and translate business requirements into data models.
• Work with the Application Development team to implement data strategies and build data flows.
• Create or update conceptual, logical and physical data models using OLTP best practices and according to Hollard’s standards.
• Recommend opportunities for reuse of data models where appropriate.
• Perform reverse engineering of physical data models from databases and SQL scripts.

Operation Data Store and Data Warehouse Development:

• Design, develop and administrate databases and data structures/models with due consideration to: the potential data queries and system feeds that will be generated from the tables; addition of timestamp and user ID columns as appropriate to identify the user and the time of the row addition or change; the use of standard abbreviations in table and column names; assignment of data types, nullability, and defaults for columns as appropriate.
• In collaboration with the DBA team implement database optimisation strategies such as indexing and partitioning.
• Creation of database objects such as tables, tables paces, synonyms, views, triggers, stored procedures, defaults, domains, and indices; maintenance and recoverability; logging and recoverability of database updates since the last backup; periodic table reorganization; structure update with change control.
• In collaboration with the DBA’s and outsource vendors, monitor and maintain database security, create security profiles and user access, grant and revoke rights and monitor usage.
• Create and update database documentation and procedures as appropriate.

Cloud Migration:

• Analyse environments to gauge cloud readiness and provide guidance on structures and processes not in a state that is ready to migrate to cloud solutions economically and efficiently.
• Develop and design a proposed cloud landscape following best practices and standards.
• Assist in the development of team skills.
• Assist in the overall cloud migration process.

Stakeholder Management:

• Liaise and engage with KAM, the off-platform partner’s BI team and DBA.
• Manager stakeholder expectation and ensure that deadlines are met (under promise and over deliver)

Required Knowledge and Experience

• Minimum 5 years' experience in the implementation, development and/or maintenance of Business Information systems in at least 3 different business environments, including financial services; of which;
• Minimum 5 years' experience in ETL integration with transactional systems;
• Minimum 2 years' experience in Cloud technologies development and integration;
• Minimum 2 years' experience specifically in relational database design, modelling, implementation and maintenance;
• Experience demonstrating in-depth understanding of Data Architecture, Database Management Technologies, Data Warehousing and Business Intelligence concepts and technologies.
• Advanced knowledge of ETL methodologies and technologies and demonstrated proficiency in Data Definition Language (DDL), Data Manipulation Language (DML), and (ETL) design and development using SQL Server tools (SQL Management Studio, T-SQL).
• In-depth knowledge and experience in SSIS.
• In-depth knowledge and experience in AZURE
• Skilled in the design and development of relational databases and data structures
• Knowledgeable on the range of options and best practices for common ETL design techniques such as change data capture, key generation and optimization.
• Skilled in the management of ETL and data quality, data profiling, metadata management and master data management, including error handling, logging & alerting, scheduling and job control.
• Advanced knowledge in working with large, complex data warehouses/data marts.
• General understanding of and proficiency in project management principles and best practices.
• Excellent SQL coding skills.
• Data Modelling skills
• Customer orientation.
• Attention to accuracy.

Personal Qualities:

• Leadership Skills
• Excellent analytical skills
• Lateral thinker/innovator
• Effective written and oral communication skills
• Teamwork and co-operation

Educational Requirements

• Matric/Grade 12
• National diploma or Bachelor's degree in Information Technology, Informatics, Computer Science , Engineering or Business Analysis
• BI Certification (e.g. Microsoft MCITP, Kimball Certification, etc.) would serve as an advantage