Azure Data Engineer – Senior Consultant at Deloitte

Job Description

• We are looking for an Azure Data Engineer to join our AI and Data practice, who is passionate about data and technology solutions, with strong problem-solving and analytical skills, tech savvy with a solid understanding of software development, driven to learn more, keeps up with market evolution and industry trends.
• You will have the opportunity to work throughout the entire engagement cycle, specializing in modern data solutions including data ingestion/data pipeline frameworks, data warehouse & data lake architectures, cognitive computing and cloud services. 

Technical Requirements for the role:

• Support AI & Data team and implement end-to-end modern data platforms in support of analytics and AI use cases
• Collaborate with enterprise architects, data architects, other ETL developers & engineers, data scientists and information designers to lead identification and definition of required data structures, formats, pipelines, metadata, and workload orchestration capabilities
• Address aspects such as data privacy & security, data ingestion & processing, data storage & compute, analytical & operational consumption, data modelling, data virtualization, self-service data preparation & analytics, AI enablement, and API integrations
• Estimate effort and mentor junior colleagues
• Participate in technical meetings with client staff, and advise client with technical option analyses based on leading practices
• Work as a data engineer on AWS but also on other technologies.
• Apply your deep knowledge of technology to drive continuous improvement.

Behavioural Competencies:

• Good communication skills, both written and verbal
• Interpersonal and relationship building skills
• Desire to develop self
• Client delivery focus
• Adaptable
• Focus on quality
• Problem solving ability
• Analytical

Qualifications

• Bachelor’s Degree (or higher) in quantitative areas such as Computer Science, Information Management, Big Data & Analytics, or related field is desired.

One or more of the following Azure certifications is preferred but experience with building solutions on cloud platforms is mandatory:

• Azure Solutions Architect – Associate/Professional
• Azure Data Engineer Associate
• Databrics

Experience:

• 5+ years’ experience in implementation of creative data solutions leveraging the latest in Big Data frameworks, supporting on-premise or Azure cloud to enable use cases in analytics and AI
• 5+ years’ experience with extraction, transformation and loading of data from a wide variety of traditional and non-traditional sources such as structured, unstructured, and semi-structured using SQL, NoSQL and data pipelines for real-time, streaming, batch and on-demand workloads
• 5+ years’ experience with data warehousing or data lakes.
• Ability to simplify complex technical concepts into easy-to-understand non-technical language in order to facilitate, communicate and interact with executives and business stakeholders, working with Agile development methods in data-oriented projects

Technical Competencies

Must Demonstrate experience in one or more services and technologies listed below:

• Database: SQL Server, NoSQL (Hbase, Cassandra or Mongo DB), Cloud Based Databases (Hive, Cosmos DB, Dynamo DB), Redshift/ Redshift Spectrum,
• Database Development: Experience Views, functions, stored procedures, Optimisation of queries, building indexes, OLAP / MDX
• Cloud:  Azure / GCP /Snowflake 
• Programming: SQL (TSQL /HQL etc), Python, Spark. UNIX & Shell Commands (Python / shell / Perl)
• Modelling: Data Vault, Kimball, 3rd Normal Form / OLAP / MDX)
• Big Data: Hadoop Platform (Cloudera / cloud equivalent), HiveQL /Spark / Ooozie / Impala / Pig), Optimising Big Data, Streaming (NiFi / Kafka)
• Data Acquisition: Pipeline creation, Automation and data delivery, Once off, CDC, Streaming

Engineering Competencies:

• Able to define a structured approach to problem solving
• Completion of data models and designs within client’s architecture and standards
• Build robust data pipelines and ETL’s using integration tools and services
• Understanding complex business environments and requirements and design a solution based on leading practices
• Ability to document design and implement solutions for client product owners
• Completion of deliverables for gaining architectural approval at client
• Understanding of DataOps approach to solution architecture.
• Solid experience in data and SQL is required
• Solid data modelling experience