Data Engineer - JHB at Hatch

A successful candidate would:

• Be involved with all technical activities within the team: estimating, architecting, designing, Agile development, code reviews, documentation.
• Introduce and guide best practices to level up our data engineering standards.
• Drive team culture through effective communication, mentoring, and participation in team growth.
• Embrace the engineering challenge and design stellar solutions that automate routine tasks.

Experience:

• Minimum of 8 years’ experience in direct development within the MS or Azure environment.
• BSc degree in Electronic Engineering, Computer Engineering, Computer Science, or equivalent from a reputable South African university preferred.
• Data engineering certifications with Microsoft is preferred.

Preferred technical competencies:

Ingestion and orchestration

• ETL Processes: Working history with ETL tools and techniques, including experience with SSIS.
• Azure Data Factory: Comprehensive understanding and hands-on experience in creating packages, organizing data flows, and constructing pipelines from various data sources, including production data from Historian, Controller, Sensor data, and on-premise databases.
• Data Management: In-depth understanding of different data types and frequencies, with the ability to ingest the information into an Azure environment.
• API and Microservices Integration: Proven experience with API integrations and employing ESB in conjunction with services and microservices architectures

Preparation, Transformation, and Enrichment:

• Data Warehouse Development: Proven ability in developing a data warehouse and robust experience in SQL.
• Technologies: Hands-on experience with DataBricks, Azure DW, Synapse, CosmosDB, MongoDB.

Store:

• Data Management: Comfort in working with various data storage types, including staging areas, blob storage, data lakes, and databases.
• Scaling: Knowledge and experience in scaling server requirements and expansion with Azure Kubernetes or equivalent functions.

Serve:

• Modeling Skills: Solid understanding of modeling concepts, including data schema and ERD (entity relationship diagrams).

Visualize:

• Business Requirements: Ability to interpret and translate business analysts' requirements into functional development needs.
• Visualization Tools: Proficiency in using PowerBI, Tableau, and other front-end development apps.