Engineer: Data X2 at Woolworths

Introduction

• Data Engineer who will assist in designing and implementing scalable and robust processes to support the data engineering capability.
• This role will be responsible for extracting and transforming massive amounts of data at scale and consolidating this data into a bigger data ecosystem.

Job description

• Design, build, and maintain scalable, automated data pipelines using AWS services like Glue, Lambda, Step Functions, EMR, and Kinesis.
• Develop ETL/ELT processes to ingest, transform, and deliver structured and unstructured data into data lakes, data warehouses, and analytical environments.
• Ensure efficient batch and real-time data processing, optimising workflows for performance, scalability, and cost efficiency.
• Implement robust data movement strategies, ensuring secure and efficient data transfer across cloud storage layers.
• Develop and maintain data transformation logic, ensuring data is structured for engineering, AI/ML, and analytics use cases.
• Implement change data capture (CDC) mechanisms to support real-time data synchronisation and streaming analytics.
• Optimise large-scale data processing workflows for performance and efficiency.
• Ensure data pipelines feed into AWS-based data lakes (S3), data warehouses (Redshift), and feature stores for AI/ML.
• Enable AI/ML and analytics teams with structured, high-quality data, ensuring seamless integration with their models and insights.
• Collaborate with engineering teams to support automated, self-service data access.
• Develop CI/CD pipelines to automate testing, deployment, and monitoring of data workflows.
• Leverage Infrastructure as Code (IaC) tools like Terraform and CloudFormation for scalable cloud data infrastructure automation.
• Implement version control, rollback strategies, and automated alerting for pipeline reliability.
• Continuously optimise data pipelines to improve performance, reduce latency, and minimise operational costs.
• Stay up to date with advancements in cloud-based data engineering and best practices.

Minimum requirements

• Bachelor’s degree in Computer Science, Business Informatics, Mathematics, Statistics, or Engineering.
• 4 - 5 years of relevant data engineering experience in cloud-based environments.
• AWS Certifications (AWS Data Analytics, AWS Solutions Architect) are advantageous.
• Strong understanding of data structures, algorithms, and software design applied to cloud-based data engineering.
• Experience working with structured and unstructured data at scale, including key-value, document, and columnar data stores.
• Proven expertise in AWS services, including Redshift, Glue, Lambda, EMR, S3, IAM, RDS, and Athena.
• Experience building and optimising cloud-based ETL/ELT pipelines for analytics, reporting, and AI/ML workloads.
• Strong programming, performance tuning, and troubleshooting skills using Python, Scala, Java, and C .
• Hands-on experience with distributed data processing frameworks, including Apache Spark and AWS Glue.
• Experience designing and implementing AWS-based data solutions, including data lakes, event-driven architectures, and API integrations.
• Experience working with streaming data solutions, including Kafka, Kinesis, and change data capture (CDC) synchronisation.
• Proficiency with DevOps-oriented data engineering, including CI/CD automation and Infrastructure as Code (Terraform, CloudFormation).
• Experience with version control systems, including Git and SVN.
• Expertise in monitoring and troubleshooting data pipelines using AWS CloudWatch.

ADDITIONAL CRITERIA

• Analytical Mindset: Demonstrates a strong analytical and problem-solving ability, capable of breaking down complex data issues and devising effective solutions.
• Collaboration and Communication: Exhibits excellent interpersonal and communication skills, with the ability to articulate complex data concepts to non-technical stakeholders. Must foster a collaborative team environment and efficiently work across different departments.
• Continuous Learning: Has a strong commitment to continuous professional development, staying ahead of the latest trends and technologies in data engineering and analytics.
• Willingness to pursue relevant certifications and training.
• Innovative Thinking: Displays innovative thinking and a proactive approach to identifying and pursuing opportunities to improve data processes and solutions. Comfortable proposing and experimenting with new technologies or methodologies to enhance data capabilities.
• Adaptability: Demonstrates flexibility in adapting to changing business needs and technology landscapes.
• Can efficiently manage multiple priorities and adapt strategies in a fast-paced environment.
• Cultural Fit: Aligns with the organisation's culture and values, contributing positively to team dynamics and company morale.
• Demonstrates integrity, accountability, and a strong work ethic.

Apply by: 3 July 2025