Senior Data Engineer at NMG Benefits

We’re looking for a curious, intelligent, and proactive cloud focused Senior Data Engineer to help us tackle complex data analytics projects end-to-end in Microsoft Azure. You will work with various teams based in multiple locations globally to deliver solutions on the Azure Cloud using core Azure cloud tools and languages. In addition, you will participate in improving recent implementations. You’re committed to delivering high quality interactions and you’re excited about making a big impact on a small team.

• You love building the best data storage, processing, and visualization solutions; apart from being functional and insightful you really want it all to look beautiful, from initial design to naming conventions, code & visualization.
• We won’t have to tell you much about data architectures, data processing and integration technologies and methodologies or business intelligence ecosystems; your expert knowledge is up to date or you can research and learn. You have deep understanding of all things data including ingestion, transformation, and consumption.
• You enjoy working in a diverse business with multiple cultures and stakeholders.
• You thrive in a fast-pace project environment where excellent collaboration and communication skills are key to success.
• When it comes to the crunch, you love the pressure of an occasional healthy deadline.
• You proactively identify opportunities for work optimisation including opportunities for automation.
• You’re fluent in English.

This position is not an infrastructure position.

KEY RESPONSIBILITIES

Data Architecture

• Participate in deep architectural discussions to build confidence and ensure customer success when building new solutions and migrating existing data applications on the Azure platform.
• Reverse engineer existing database data models, manage and maintain existing and new logical and physical data models.

Data Engineering (Development)

• Collaborate with functional group leaders and engineering team(s) to gather and analyse business and technical data requirement needs and understand how data is collected, analysed & utilised to design and implement the management, monitoring, security, and privacy of data using the full stack of Azure data services.
• SQL server development and coding complex functions, stored procedures, triggers, indexes, queries or ad-hoc analyses, and views (using T-SQL).
• Design and develop Azure Data Factory ETL, ELT or ingestion processes that will transform a variety of on-prem and cloud (structured and unstructured) data sources into SQL databases or data warehouses or integration solutions, including REST APIs, Event Drive or Queue based integrations.
• Design and build Microsoft Azure functions to optimize data extractions and ensure data validation, cleansing and merging forms a critical part of data processing solutions.
• Ensure that data services securely and seamlessly integrate with other data platform technologies or application services such as Azure Cognitive Services, Azure Search, or even bots.
• Enhance existing or build new SQL Server Analysis Services solutions, tabular models or OLAP cubes used in the business intelligence ecosystem and develop Multi-Dimensional Expression (MDX) queries to extract data from OLAP cubes for reporting and analytical purposes.
• Enhance existing or build new enterprise or departmental business intelligence solutions, inclusive of Power BI.
• Adhere to or recommend best practice cloud services, database or data engineering, and identity standards and perform team and 3rd party code reviews in accordance with such standards.
• Build prototypes or pilots using new technologies.
• Present solutions and recommendations to stakeholders.

Testing

• Perform unit, integration, or system testing (automated or otherwise) on all developed code and / or system components through stringent routines and procedures to ensure accuracy and solution integrity and that solutions run smoothly with optimum operational efficiency, ensuring all solutions will meet SLAs & performance criteria.
• Provide feedback on solutions’ usability, features, and design based on results of testing.
• High focus on performing your own data reconciliations during testing phases to ensure that your development work has been completed successfully prior to deployments for user acceptance testing.

Production Support and Maintenance

• Provide exceptional support by applying critical thinking skills to troubleshoot, determine the cause of failure, and quickly restore failed components or processes when they occur; Diagnose and remediate resource contention issues and failures in application logs.
• Participate in an on-call rotation with the team when necessary, specifically related to customer-facing digital cloud solutions.
• Report and escalate issues to 3rd party vendors if necessary.
• Conduct monthly reviews of incidents and service requests, analyse, and recommend improvement in quality and work with the internal team on identifying pain points in existing Azure deployments and configuration and ways to alleviate them.
• Working experience in hardening cloud production environments for error handling, fault tolerance, self-healing, monitoring and incident alerting and recovery beneficial.
• Monitor connections and locks and performance of SQL instances to track historical peak load on servers and proactively working on performance tuning and writing queries for front-end applications.
• Manage, monitor, and ensure the security and privacy of data to satisfy business needs.

Other Critical Deliverables:

• Legacy Migrations: Migrating on-prem SQL instances and legacy SAS datasets to Azure SQL in development, test, or production environments.
• Training and Mentorship: Provide technical training and mentoring to other teams and team members and organize and execute training sessions for the user base of in-house developments for new workflows, procedure recommendations, the availability of data in operational data stores as well as the data warehouse and how to consume the data effectively in business intelligence and analytics tools within the organization.
• Documentation: Originate and maintain documentation for new and existing solutions throughout the solutioning life cycle, covering all applicable functional areas, such as bugs, change requests, operational policies and procedures, solution designs, integration and API specifications, technical specifications, test plans and test results, production control (and / or job scheduling), security administration, TSQL code and logical as well as physical data models where applicable.
• Our values: Living the NMG values of Collaboration, Curiosity, Go for It, and Make it Count in all that you do.

YOUR EXPERIENCE & CAPABILITIES

The successful candidate for this role will be able to demonstrate:

• Strong organisational abilities and high attention to detail.
• The ability to thrive within a small team whilst also working independently.
• Agility in approach, reacting positively to change and shifting priorities.
• Effective communication skills and the ability to collaborate cross-group or cross-geo.
• Working well under pressure with excellent time management skills.
• A passion for technical excellence and a flair for user experience and design.
• Excellent analytical, process design and problem-solving skills.
• Resourcefulness and troubleshooting aptitude.
• Ability to communicate technical needs and solutions with non-technical staff and comfortable performing component demonstration for key business stakeholders and project managers.

Technical Capabilities should encompass the following:

• Experience in multiple or all Azure components, including: API Management, Event Hubs, Data Factory, Functions, Resource Manager Templates, Storage Accounts, Notifications Hub, Key Vault, DevOps, Data Lake Stores, Data Lake Analytics, Synapse Analytics, Databricks, HD Insight, SSAS, SQL Database or similar cloud infrastructure (5+ years’ experience and deep expertise in data engineering as applied to Azure preferred), including Visual Studio as applied to SSAS development.
• In-depth knowledge of standard concepts, practices and procedures related to database modelling (logical and physical) and management, concepts of data lakes, data warehousing and data marts as well as legacy migrations to cloud services.
• Advanced Business Intelligence experience, understanding of BI areas and reporting using SQL, SSAS, Tabular Models and Power BI, including proactive identification of issues and coordination of resolutions.
• Infrastructure automation for continuous integration and continuous deployment of technical solutions leveraging Azure Services and Features.
• Modern version control Git, SVN, TFS, etc.

Experience that will make you stand out:

• Being able to conceptualize the full project life cycle.
• Managing and optimizing cloud costs associated with development.
• QA testing.
• Applicable Azure certifications including for example Implementing an Azure Data Solution, Designing an Azure Data Solution, Designing and Implementing Microsoft DevOps Solutions.
• Familiarity with the Technology stack available in the industry for data management, data ingestion, capture, processing, and curation as well as metadata management: data governance, data quality, master data management, lineage, data cataloguing, etc.
• Knowledge of self-hosted integration run time will be an added advantage.
• Hands-on experience in scripting languages such as Python.