Data Engineer (Python, PySpark, Azure/AWS) (CPT) at Datafin Recruitment

DUTIES:

• Own data products from ingestion through to trusted consumption, ensuring data is reliable, accessible, and fit for purpose.
• Design, build, and operate scalable ELT and ETL pipelines across structured and unstructured data sources.
• Develop and maintain modern cloud-based data platforms using industry-standard Engineering practices and tooling.
• Implement data quality, governance, and observability capabilities to ensure trust in data products.
• Deliver data solutions through CI/CD pipelines and Infrastructure-as-Code, contributing to a robust and maintainable Engineering ecosystem.
• Evaluate and apply emerging technologies, including AI and LLM-based tooling, to improve Engineering productivity and business outcomes.

REQUIREMENTS:

Qualifications –

• Degree in Computer Science, Engineering, Mathematics, Science, or a related quantitative field.

Experience/Skills –

• At least 4 years of experience building, deploying, and operating data pipelines in production environments.
• Strong Python and PySpark skills, with a solid understanding of modern Data Engineering practices.
• Hands-on experience with a major cloud platform such as Azure, AWS, or GCP.
• Experience building and maintaining CI/CD pipelines using tools such as Azure Pipelines, GitHub Actions, or Jenkins.

ATTRIBUTES:

• The ability to build and maintain meaningful relationships.
• Able to ‘approach and own’ and continuously looks for opportunities to develop.
• A strong belief in doing the right thing.
• Can recognise and embrace change.