Data Engineer at DigiOutsource

Purpose:

This is an excellent opportunity for an individual who is looking to gain new skills. We are committed to creating a culture of learning and regularly run proof of concepts, provide internal demos, manage workshops, and attend external conferences. Our data team has a wide range of skills including development for highly transactional RDBMS systems, data warehousing, data science and cloud technologies and we are always working to raise everyone’s game. 

Duties will include, but not limited to:

Operating Duties

• Work as part of an agile Data Engineering Team

• Development of greenfield projects in Azure

• Develop and maintain on-premises RDBMS/DW systems

• Create robust ETL services and real-time data pipelines

• Build and maintain Continuous Integration and DevOps pipelines

• Test-driven development and pair programming with colleagues

• Demo work to both technical and non-technical stakeholders

• Create documentation and training material for the solutions you deliver

Non-Negotiable Requirements:

• Highly skilled and experienced in using MS SQL Server in a highly transactional environment

• Able to understand query plans and identify performance bottlenecks

• Can communicate an understanding of transactions, locking and isolation levels

• Experience developing ETL processes

• Exposure to Azure cloud technology

• C#, .Net, Visual Studio

• Git, Azure DevOps

• Communicate effectively with both technical and non-technical stakeholders

Advantageous Requirements:

• Building Continuous Integration/Deployment pipelines

• Implementing Big Data Storage and Processing solutions in Azure (ARM Templates, Storage Accounts, Data Lake Storage, SQL DB, Cosmos DB, EventHubs, Function Apps, Log Analytics, Data Factory, Databricks, Polybase)

• Implementing Security, Disaster Recovery, High Availability, Auditing, Monitoring and Alerting solutions in Azure

• Automating tasks in Azure using Powershell / CLI

• Predicting costs and optimizing spending in Azure

• Test/Business-Driven Development

• Message Queues (RabbitMQ, Kafka)

• Terraform

• Python, PySpark, Scala

• Tableau, PowerBI, SSRS

Job Competencies:

• Develop rich applications using the latest technology.

• Architect efficient and reusable systems that drive complex applications.

• Have an active role in all features and design decisions, defining system architecture.

• As Technical Lead be involved in defining development best practices and applying them in a large, distributed Development Team.

• Ability to maintain existing code base with variant designs and implementations and strive towards a better tomorrow.

• Have an active role in planning and agile process with team.

• Transfer knowledge to team and upskill team members

Personal Competencies:

• Adaptability

• Ownership & Accountability

• Initiating Action

• Resilience

• Team Orientation

Integrity and Innovation