BI_BA Developer (Databricks, Spark SQL, PySpark) (JHB) at Datafin Recruitment

ENVIRONMENT:

• A dynamic Healthcare Platform seeks a Business Intelligence/Business Analyst Developer to fill a hands-on role focused on building and maintaining reporting and analytics solutions on a modern lakehouse platform.
• The primary focus will be the development of management reporting and operational analytics for key data partners, alongside contributing to the broader data platform.
• The successful candidate will work to specifications agreed with the General Manager and associated data partners and deliver against a defined backlog of reporting, modelling, and pipeline development work.
• You will combine strong SQL and BI development capability with the analytical mindset needed to translate business requirements into well-modelled, accurate, and maintainable data products.
• Applicants will require a Bachelor’s Degree in IT/ Computer Science/Information Systems or related field with 3-6 years’ experience and proficiency with Databricks, Spark SQL, PySpark, T-SQL, Power BI, Python, Git & Advanced Excel skills including Power Query and Power Pivot.

DUTIES:

BI Development & Reporting –

• Design, build, and maintain management and operational reports using Power BI.
• Develop semantic models and DAX measures against the Databricks lakehouse.
• Build dashboards and KPI views for management stakeholders.
• Tune Power BI datasets and reports for performance, accuracy, and usability.
• Maintain and extend existing reports, extracts, and dashboards as requirements evolve.
• Validate report outputs against source data and resolve discrepancies.

Data Pipeline Development on Databricks –

• Develop and maintain notebooks and workflows across the medallion architecture.
• Write Spark SQL and Python to transform pharmacy claims, dispensing, and product data.
• Support Lakeflow Connect ingestion from Azure SQL and other source systems.
• Apply Unity Catalog governance, access controls, and data classification standards.
• Optimise notebook and workflow performance for large datasets.
• Contribute to platform documentation, code reviews, and version control discipline via GitHub.

Business Analysis & Requirements –

• Work with the General Manager and internal stakeholders to clarify reporting and analytics requirements.
• Translate business questions and specifications into clear data and reporting designs.
• Document data definitions, business rules, and report specifications for ongoing maintenance.
• Conduct ad-hoc analysis to answer specific operational and commercial questions as required.
• Validate deliverables internally before release and incorporate feedback iteratively.

Data Quality & Documentation –

• Investigate and resolve data quality issues across the pipeline.
• Maintain documentation of models, measures, pipelines, and business logic.
• Apply consistent naming conventions and coding standards across all development work.
• Participate in peer review of code, models, and reports.

REQUIREMENTS:

Qualifications –

• Bachelor Degree in Information Technology, Computer Science, Information Systems, Data Science, Statistics, or a related field, or equivalent demonstrable experience.

Experience/Skills –

• 3-6 Years of experience in Business Intelligence Development, Data Analytics, or a closely related role.
• Demonstrated experience with Power BI in a production environment.
• Working experience with Databricks or a comparable lakehouse platform.
• Proven SQL development experience on a relational database platform.

Required Technical Skills –

Databricks & Lakehouse Platform:

• Working experience with Databricks for data processing and analytics.
• Proficiency in Spark SQL and PySpark for data transformation.
• Understanding of the medallion architecture pattern.
• Familiarity with Unity Catalog, Delta Lake, and lakehouse concepts.
• Experience developing and maintaining production Databricks notebooks and workflows.

SQL & Data Modelling:

• Strong proficiency in SQL and T-SQL, including complex joins, window functions, and query optimisation.
• Experience with dimensional modelling, star schema design, and semantic layer construction.
• Understanding of data warehousing concepts and modern lakehouse equivalents.
• Experience with Azure SQL Database as a source system.

Business Intelligence & Reporting:

• Strong proficiency in Power BI for dataset design, report development, and DAX measure authoring.
• Experience designing semantic models that connect Power BI to a lakehouse backend.
• Ability to design dashboards that match the needs of executive, manager, and operational user personas.
• Advanced Excel skills including Power Query and Power Pivot.

Programming & Tooling:

• Working proficiency in Python for data processing and notebook development.
• Comfortable with Git and pull request workflows for version control.
• Familiarity with CI/CD concepts for data and BI deployment.

Cloud Platform:

• Working knowledge of Azure data services, particularly Azure SQL and Azure storage.
• Understanding of cloud security and access control fundamentals.

Preferred to haves –

• Pharmaceutical or healthcare domain knowledge (highly preferred): Direct experience with pharmaceutical, pharmacy, or healthcare data including claims, prescriptions, dispensing records, or market research data.
• Databricks certification (Data Engineer Associate or Data Analyst Associate).
• Microsoft Power BI Certification.
• Azure Data Certifications (Azure Data Engineer Associate or Azure Data Fundamentals).
• Familiarity with pharmaceutical data products.
• Exposure to legacy SQL Server BI stack (SSIS, SSRS, SSAS) for transition and integration scenarios.
• Knowledge of POPIA and privacy-preserving analytics concepts as applied to healthcare data.

ATTRIBUTES:

• Strong analytical and problem-solving abilities.
• Strong attention to detail and a quality-first mindset on data accuracy.
• Clear written and verbal communication, particularly in documenting work and explaining technical detail.
• Ability to manage multiple priorities and meet delivery commitments.
• Self-directed, with the ability to scope and progress work independently once requirements are clear.
• Strong organisational skills and disciplined documentation habits.