Jobs Career Advice Signup

Send this job to a friend


Did you notice an error or suspect this job is scam? Tell us.

  • Posted: Aug 3, 2021
    Deadline: Not specified
  • Integrove is a Johannesburg based boutique consultancy focusing on technology architecture and integration in the enterprise. We help organizations to enable integration and technology architectures that provide the foundation for their application, mobility and cloud strategies.
    Read more about this company


    Senior Azure Data Engineer / Databricks Developer / Python

    About the job

    The Data Engineer will proactively work with the solution architects, business owners, business representatives, other systems team members to understand business and governance requirements and to implement solutions to address these requirements. The Data Engineer will design, configure, develop and deploy data ingestion pipelines and transformations.  You will develop data pipelines using ARM templates so that they can be deployed automatically to other environments/brought back up. You will develop data transformations in Azure Databricks using Python and on Azure SQL using T-SQL and deployed using ARM templates. These data transformations will be used to combine and curate data as well as transforming it into dimensions and facts and therefore a strong knowledge of standard BI concepts is mandatory.


    Duties and Responsibilities

    • Writes technical designs

    • Installs integration runtime on servers to Integrate with Azure

    • Develops / configures data ingestion / transformation pipelines

    • Develops / configures data ingestion / transformation maintenance routines (e.g. concatenation of files, pipeline monitoring, reconciliation)

    • Deploys data ingestion / transformation pipelines

    • Develops & deploys Microsoft Power BI dashboards

    • Establishes and monitors data quality initiatives

    • Works with the Business data owners to document data definitions and business rules

    • Identifies areas for data quality improvements and helps resolve data quality problems

    • Develops data visualisations using Microsoft Power BI


    Skills & Experience

    • B.Sc. or Business degree or equivalent

    • 5+ years of experience in elements of data governance: quality, analysis, administration, architecture

    • Experience working as a Data Engineer on data warehousing solutions and cloud technology implementations

    • Experience working with data visualisation tools, e.g. Power BI, Tableau, etc.

    • Detailed knowledge and experience working with the following:

      • Microsoft Azure Data Factory V2 – including connectors to SQL Server, SAP and custom connectors. Other connectors may also be used depending on final source systems selected

      • Microsoft Azure Data Lake Store Gen2

      • Microsoft Azure ARM Templates / Azure Blueprints

      • Databricks (PySpark) – including integration of Azure Data Factory with Databricks

      • T-SQL

      • PowerShell or equivalent

      • Data formats (AVRO and Parquet)

      • Strong knowledge of BI concepts, e.g. enterprise data models, ER mapping, data warehouse design (dimensions & facts)

      • Azure DevOps

      • Integration Runtime

    • Superior attention to detail

    • Strong analytical, problem solving and organizational skills

    • Ability to compare technologies and make recommendations to senior members of the team.  

    Method of Application

    Interested and qualified? Go to Integrove on to apply

    Note: Never pay for any training, certificate, assessment, or testing to the recruiter.

  • Send your application

    View All Vacancies at Integrove Back To Home

Career Advice

View All Career Advice

Subscribe to Job Alert


Join our happy subscribers

Send your application through

GmailGmail YahoomailYahoomail