Senior Data Engineer (Machine Learning) at Black Swan Data

What You’ll Do

We are looking for a talented developer to join our team. The role involves building and maintaining high performance big data machine learning based models and pipelines to facilitate all our SaaS product for some of the world’s leading brands. You'll be working closely within a team of developers and data scientists distributed remotely across the UK, South Africa and Hungary.

To apply for the role, you’ll need

• To be the type of developer that can pick up a new technology quickly and deliver features in a highly agile manner.
• Experience writing testable functional Python in a production grade system.
• To have used Apache Spark in a production system using Python
• Having experience in Spark orchestration tools such as Apache Airflow
• Experience of using a cloud platform to architect and build data pipelines.
• To easily navigate the administration of a Hadoop cluster on a cloud platform such as Databricks
• Will have worked closely with Data Scientists helping to productionise Machine Leaning methodologies using PySpark working with large RDD’s
• Working with the full ML lifecycle using MLFlow and/or Seldon
• Proven track record working on with a wide range of NLP problems such as topic modelling, relevancy and disambiguation is essential.
• Experience working with social data is also desirable.

What You Will Learn From The Role

• We try and automate as much as possible to streamline our processes. This includes continuous integration and deployment.
• Quality is very important to us, and through automated testing we strive to keep the quality of our systems as high as possible.
• Working with large sets of data and using distributed databases.
• Agile delivery process within squads made up from multiple skillsets.
• The potential for learning and working in another language like Java, Scala or Python.
• End to end exposure to a cloud platform.