Data Engineer - Cape Town at Pepkor Speciality

KEY RESPONSIBILITIES

• Data Exploration and data quality assurance
• Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality
• Acquire data from primary or secondary data sources and maintain databases/data systems
• Filter and “clean” data by reviewing other reports and performance indicators to locate and correct code problems
• Development, construction, and maintenance of data architectures.
• Conducting testing on large scale data platforms.
• Handling error logs and building robust data pipelines
• Ability to handle raw and unstructured data.
• Provide recommendations for data improvement, quality, and efficiency of data. 
• Ensure and support the data architecture utilized by data scientists and analysts.
• Development of data processes for data modeling, mining, and data production
• Analyze data through descriptive statistics
• Communicating results with the team and business using data visualization 
• Design and build production grade minimum viable products such as dashboards and published notebooks for business users, linked to the feature stores, scoring engines and optimisation models.
• Feature engineering
• Standardization or Scaling of the data
• Create Value Transformation like Square or Cube
• Design and build the feature stores for model training
• Operationalize feature stores for model training
• Work with management to prioritize business and information needs
• Locate and define new process improvement opportunities
• Presentations to management
• Identify the goal and frame the business problem.
• Gather information on resources, constraints, assumptions, risks.
• Prepare Analytical Goals and Benefit to business
• Understanding business and information flow  
• Drive best practice, continuous improvement and innovation within the Data Engineering function
• Evaluate new technologies and products to determine feasibility and desirability of incorporating their capabilities into data engineering

JOB INCUMBENT REQUIREMENTS

• BSc in Mathematics, Economics, Computer Science Management or Statistics, National Diploma in Information Technology, BEng. 
• 5 years of relevant experience within the field of data engineering/software development.
• Knowledge of core computing concepts like programming and algorithms to build robust data systems.
• Knowledge of application development and working of APIs;
• 3- 4 years of programming
• Proficient in Visualisation
• Cloud tools: (preferably on GCP) : Big Query, Composer/Airflow, AI Platform, Kubernetes, Dataflow, Pub/Sub, Cloud (Functions, Scheduler, SQL, Storage), Compute Engine, Monitoring
• Knowledge of the following will be advantageous: Git workflow, Agile management (JIRA, Shortcut, Trello), Slack, CI/CD, Docker, SQL, Python, JavaScript, Google Workspaces, Visualization tools: QlikView / LookerStudio / Grafana