Data Engineer at IQbusiness

Personal competencies that we are looking for: 

• The ability to solve problems.
• The ability to rotate around a problem, to see if solutions can be gained in different ways.
• The ability to work in an ever changing, unstructured environment.
• The ability to work as part of a team, with vastly differing skill sets and opinions.
• The ability to contribute ideas to the quorum.
• The ability to mentor and provide guidance for other team members.
• A systems approach to thinking, as opposed to a siloed approach. You need to understand how your work affects the greater system.
• The ability to work without supervision and take accountability for the work you deliver.
• The ability to liaise with clients, the ability to ignore extraneous data and extracting the core requirements.

RESPONSIBILITIES:

(As the field of data engineering is very broad, you may be required to develop API endpoints for the consumption of data, data pipelines, highly distributed, high availability data processing pipelines in an effort to satisfy the need of the Client, Data Analysts or Quantitative Analysts.)

• Providing a superior customer experience by building and maintaining relationships, working at customer offices, keeping data confidential
• Design and develop data solutions
• Produce solutions for both real-time and batch environments
• Test your work to ensure compliance
• Solve problems, and do root cause analysis
• Prioritize your workload
• Assist in other areas relating to your data function, such as data modelling, data and business analysis
• Interact with a variety of individuals: senior, junior and technical business representatives
• Provide guidance to colleagues or manage a team
• Uphold the IQbusiness values, which are to care, learn & share; innovate; deliver as well as to build, enhance and maintain relationships.

REQUIREMENTS:

• 4 - 10 years experience in Data Engineering environments and technologies
• Cloud exposure and/or project experience – Microsoft Azure, AWS, etc
• Extensive experience in Microsoft Azure technologies
• Extensive experience in reporting and dashboarding technologies such as Power BI and QlikView
• Experience using Azure DevOps and CI/CD frameworks
• Exposure to logical and physical data modelling
• Agile and waterfall project management
• Relevant Qualifications Information Technology
• Additional MS, Data and/or Cloud certifications are beneficial.

Your knowledge about the following concepts, is valuable to the team:

• Data modelling
• Relational data modelling in traditional database management systems
• Coercing unstructured and semi-structured data into structured form
• Data pipeline knowledge (extracting and transformation)
• Data transformation knowledge for reporting and analytics purposes

Some technical competencies that we are looking for:

• Writing SQL queries, that can relate, transform and aggregate data from many different sources
• Experience with writing data pipelines that move and transform data
• Working knowledge of a general purpose programming language like Python would be advantageous
• Working knowledge of Microsoft Azure technologies such as Azure Data Factory, Azure SQL, Databricks, Synapse Analytics.
• Working knowledge of Storage Repositories and Data Lakes