Databricks Data Engineer at Intelligent Debt Management

• The Intelligent Debt Management (IDM) Group, South Africa’s largest and multi-award winning debt management company, has a position available for a Data Engineer. 
• This individual will join the Data Team and will be accountable for data engineering design, data ingestion and support across on-premise and cloud-based platforms.  

Requirements 

Candidates must be able to demonstrate:   

• 3 years experience as a data engineer or in a similar role, designing and implementing data pipelines, data integration and ETL processes. Practical experience in Azure, Azure Data Factory and Azure Databricks is preferred. 
• Proven experience in Python/PySpark and SQL/SparkSQL programming skills. 
• At least 3 years of experience in Information Technology – specifically in technical data roles e.g. data engineering, data analytics or data science. 
• Knowledge of data management and storage solutions, including SQL Server Database, Azure Data Lake Storage or similar. 
• Having good communication skills and being able to operate in a team environment. 
• Excellent attention to detail and problem-solving skills related to data, data structures and algorithms. 

Key responsibilities will include: 

• Contribute to the architecture and design of the IDM Data Platform and ingestion patterns. 
• Implement and support the scalable data solutions on the Microsoft Azure cloud platform, including Azure Databricks. 
• Develop, maintain and optimise robust production data pipelines for data ingestion, processing, and transformation to support analytics, machine learning and other downstream data-driven services. 
• Ensure data security, quality and integrity across all of the pipelines and related data processes. 
• Collaborate with the data product owner, analysts, data scientists, and business stakeholders to understand data requirements and implement effective data workflows. 
• Assist in data initiatives and support technical teams including architects, analysts, software developers, and data scientists. 

Qualifications: 

• Bachelor's degree in Computer Science, Engineering, or a related field. 
• Certifications in Azure, Microsoft Fabric Data Engineering, Azure Databricks Data Engineering or similar or equivalent practical experience. 

Deadline:31st March,2026