Senior Data Engineer at Iress

• The Senior Data Engineer is responsible for designing, developing and maintaining high-performance, secure, and compliant data software/pipeline solutions for the wider Iress South Africa business.  Success in this role involves designing and implementing reliable, maintainable, and regulation-compliant scalable data processing solutions that align with business requirements and industry standards. The ideal candidate has strong technical expertise in financial markets and software and data engineering, ensuring systems meet correctness, performance, regulatory compliance, and availability expectations.

Some of the awesome things you’ll be involved with: 

• Design and develop high-quality data processing solutions for financial markets applications.
• Ensure correctness and accuracy of financial and other data processing.
• Implement and maintain data processing systems that meet business, compliance, and regulatory standards.
• Optimize performance and availability of data processing systems.
• Deliver solutions in short cycles, ensuring rapid feedback and the ability to course-correct as needed.
• Collaborate with key stakeholders to ensure reliability, security, and operational resilience.
• Conduct code reviews, testing, debugging, and troubleshooting to ensure software and data quality and system stability.
• Document technical specifications, system architectures, and deployment procedures.
• Mentor and support junior engineers, fostering continuous learning and knowledge sharing.
• Work with infrastructure and cloud teams to ensure smooth deployment, monitoring, and operational continuity of production systems.
• Contribute to architectural decisions and technical strategy with a focus on long-term maintainability and compliance.

What you will bring: 

Qualifications and Certifications

• Bachelor’s degree in Computer Science, Engineering, or a related field (required).
• 5+ years of experience in software development, preferably within financial markets, trading, or regulatory-compliant systems.
• 5+ years of experience designing and maintaining scalable data pipelines.
• Proven experience as a Data Engineer or similar role, with a strong understanding of data engineering principles, methodologies, and best practices.
• Proficiency in programming languages such as Python, Java, or Scala, and experience with data processing frameworks such as Apache Spark, Apache Flink, or Apache Beam.
• Experience with relational databases (e.g., PostgreSQL, MySQL), NoSQL databases (e.g., MongoDB, Cassandra), and data warehousing technologies (e.g., Snowflake, BigQuery).
• Solid understanding of data modeling concepts, data warehousing architectures, and ETL processes.
• Strong analytical and problem-solving skills, with the ability to design and implement scalable data solutions to address complex business requirements.
• Experience working in regulated environments with a focus on auditability and data integrity.
• Certifications in relevant technologies or methodologies are a plus (e.g., AWS Certified Developer, Financial Markets Certifications).

Professional Skills and Competencies

• Software Development & Architecture: Designs and builds reliable, maintainable, and well-structured scalable data processing systems using modern programming languages and frameworks.
• Testing & Quality Assurance: Implements robust testing strategies, including unit, integration, regression, and auditability testing.
• Performance & Availability Optimization: Enhances system reliability, uptime, and responsiveness under expected loads.
• Risk & Compliance Management: Ensures adherence to financial regulations, security policies, and industry standards.
• Data Governance & Security: Experience with data privacy, compliance, and governance.  Understanding of data lineage, metadata management, and observability.  Experience implementing RBAC (Role-Based Access Control) and security best practices.
• Incident Management & Troubleshooting: Diagnoses and resolves production issues to maintain system stability.
• Collaboration & Communication: Works closely with product, engineering, security, and SRE teams to develop and maintain resilient software and data processing systems.
• Iterative Delivery & Continuous Improvement: Adopts a short-cycle development approach for rapid feedback and course correction.
• Technical Leadership & Mentorship: Contributes to system design decisions and provides technical guidance to junior engineers.
• Industry Awareness & Continuous Learning: Keeps up with financial markets trends, software engineering advancements, and best practices in cloud and security.

Technology Stack

• Languages: Python, Java, Scala, scripting/automation languages (Bash, Powershell, Make etc.)
• Data processing frameworks: Apache Spark, Apache Flink, Apache Beam
• Databases: Relational databases (MS SQL, PostgreSQL, MySQL), NoSQL databases (MongoDB, Cassandra)
• Data platform: Databricks
• Hosting: AWS
• CI/CD Tools: GitHub, Buildkite, Artifactory
• IaC Tools: Terraform