Data Engineer (Python & SQL) at Datafin Recruitment

DUTIES:

• Architect, build, monitor and maintain data integrations and ETL pipelines.
• Ensure data quality and implement tools and frameworks for automating the identification of data quality issues in data source pipelines.
• Investigate, document and resolve issues reported internally or by clients.
• Write custom scripts to fix or import data for clients with specific needs.
• Implement sustainable data pipeline solutions, viz. create admin tools and scripts to address repeat support issues or requests.
• Communicate problems and workarounds to our internal support team in a simple, concise manner they can use with customers.
• Assist with answering Tier 2 Support Questions.

REQUIREMENTS:

• 3+ Years experience with integrating data and working with data sources.
• 3+ Years programming experience in Python.
• Proficient in SQL.
• Experience with data orchestration tools such as Apache Airflow.
• Experience working with AWS. Alternatively, experience with GCP or Azure would be beneficial.
• Ability to troubleshoot issues and determine and resolve the root cause of an issue.

Advantageous

• Experience with Docker or Kubernetes.
• Understanding of best practices with regards to web applications and OOP.
• Understanding of system integration design patterns.
• Mathematical proficiency.
• Exposure to these libraries or frameworks: NumPy; Pandas; PyJanitor; Great Expectations; Pandera; WhyLogs.

ATTRIBUTES:

• Professional communication skills with the ability to engage directly with clients.
• A client-focused mindset, as finding solutions that make clients happy should be your number one priority. 
• The ability to work independently.