Data Engineer at Allan Gray Proprietary Limited

Job Summary

• We are looking for an exceptional Intermediate to Senior Data Engineer to join our team. In this pivotal role, you will develop and implement the technical direction of the data engineering space. You will collaborate with data analysts, Data Scientists and senior IT members to deliver business insights and decisions that add value to Allan Gray’s client proposition.
• We have adopted a Microservices architecture with RESTful API’s predominantly written in Scala, with many built in Go, and most of the data science services written in Python. Our Microservices are built and deployed on an elastic containerized infrastructure supported and managed on premises through Kubernetes. Over time our solutions will increasingly be based in the Cloud. As a team, we apply best practises in user experience, DevOps, web application architecture, solution design, scaling and testing.

Job Responsibilities

• Lead and guide the technical vision, leveraging data engineering principles and tools to drive innovation and excellence within the team.
• Build and maintain software that leverages existing data architecture and tooling (databases, APIs) to allow data analysts to use predictive modeling that delivers business insight
• Collaborate with senior IT architects to design, build and maintain data infrastructure and pipelines to source, store and analyse internal and external data where required
• Design clean and simple solutions which are aligned to overall architectural decisions and ensures seamless integration with existing systems.
• Ensure that elegant, robust and secure code is written and maintained.
• Interpret requirements and implement technical solutions
• Improve and optimize the performance of existing systems
• Contribute and drive technical decisions
• Ensure that unit, integration and acceptance tests for all components are written
• Ensure that build automation scripts are written
• Assist with analysis where required

Requirements

• 5-8 years working experience as a Data Engineer, developer or related software development role
• Experience in designing, building and maintaining technical solutions
• Experience in architecting databases and building data pipelines/ETL processes would be advantageous
• Familiarity with DevOps; continuous integration/deployment workflows and tools (e.g., Gitlab)
• Strong understanding of software development process
• Proficiency in modern programming language (Python preferred but not required)
• Strong interest in working with data
• Experience in system integration, distributed computing and building enterprise-wide systems
• Knowledge of professional software engineering practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes and testing

Education

• Degree in Computer Science, Computer Engineering, Information Science or related technical discipline