Data Engineer at Unitrans

Job Advert Summary    

• We are seeking a Data Engineer specializing in enterprise data modelling to work at the forefront of the design and optimization of our data architecture in a dynamic logistics environment. You’ll be designing and developing comprehensive data models that unify data from a wide array of sources: internal company systems, ERP systems, IoT sensors providing live data feeds, and third-party data from partners and suppliers.
• Your focus will be on building and maintaining the platform for real-time insights, predictive analytics, and machine learning by designing efficient data models, implementing semantic layers, and centralizing business logic. Your role will involve integrating, modelling, and structuring this data in a way that maximizes its utility for business intelligence, real-time reporting, and machine learning applications. You’ll play a key role in building and maintaining our cloud- based data lake, using Azure Synapse Analytics and following a medallion architecture that ensures clean, organized, and accessible data at every stage. Collaborating closely with analytics team, you’ll translate complex business requirements into scalable, high-performance data solutions that enhance decision-making and operational efficiency.
• You will develop indexing strategies, optimize data models for performance, and support the creation of feature stores for analytics and machine learning. This is a unique opportunity to leverage cutting-edge cloud technologies and advanced data modelling techniques to improve the efficiency of supply chain operations and deliver measurable business impact.

Minimum Requirements    

Experience and Qualification

• Bachelor's degree in Computer Science, Information Systems, or a related field OR proven capability in working with enterprise data systems.
• Azure Data Engineer Associate (DP-203) preferred
• Experience with Azure cloud services, particularly Azure Synapse
• Strong understanding of cloud data solutions and architectures, and data modelling techniques, including dimensional and relational modelling.

Skills and Attributes

• Highly SQL proficient, with experience in building and optimizing complex queries for data
• Highly Python proficient
• Process design and implementation
• Proficiency in designing and maintaining data schemas and keys, including primary, foreign, and composite
• Familiarity with data modeling best practices in cloud data platforms and indexing
• Familiarity with on-premises SQL Server databases and familiarity with migration processes to cloud
• Familiarity with Qlik Sense or similar reporting and analytics
• Familiarity with enterprise systems and working with multiple
• Strong problem-solving skills and a demonstrated ability to learn new technologies
• Proactive approach to problem solving
• Ability to function within various teams and environments, but also work independently
• Excellent communication skills

Duties & Responsibilities    

• Monitor, maintain and improve data pipelines and data models and schemas in Azure Synapse
• Design, develop, and optimize data models in a medallion architecture to support analytics and reporting
• Work with third party consultants to implement data lake
• Build and maintain data pipelines to facilitate the smooth flow of data between various systems and
• Ensure consistency in data model design, including proper use of keys, indexing, and semantic layers to improve query performance.
• Provision datasets for consumption side users, focusing on consistency and data
• Migrate business logic from Qlik Sense to Azure Synapse
• Migrate on premises data systems to the Azure
• Test and maintain new and existing API
• Contribute to and expand a data taxonomy to ensure consistency and clarity in data
• Implement data security protocols across the entire data
• Focused improvement initiatives of existing processes, systems, data,
• Focused improvement initiatives of cross-functional interactions within the organisation