Senior Azure Data Engineer / Databricks Developer / Python at Vito Solutions

JOB OVERVIEW:

As a Senior Azure Data Engineer, you will play a crucial role in designing, developing, and maintaining our data infrastructure on the Azure cloud platform. You will collaborate with cross-functional teams to implement scalable and efficient data solutions, ensuring data integrity, security, and availability. The ideal candidate will have a strong background in Azure data services, ETL processes, and data modeling.

DESCRIPTION OF THE POSITION:

Azure Data Platform Development:

• Design, develop, and maintain data solutions on the Azure cloud platform, leveraging services such as Azure Data Factory, Azure SQL Database, Azure Databricks, and others.
• Implement efficient ETL processes to extract, transform, and load data from diverse sources into the Azure data platform.

Data Modeling:

• Create and maintain data models, ensuring they align with business requirements and best practices for performance and scalability.
• Optimize data structures and schemas to support efficient querying and reporting.

Data Integration:

• Integrate data from various sources, including on-premises and external systems, into the Azure data platform.
• Collaborate with data scientists and analysts to enable seamless access to relevant data for analysis and reporting.

Performance Optimization:

• Monitor and optimize the performance of data pipelines and queries to meet defined SLAs.
• Troubleshoot and resolve issues related to data processing and storage.

Data Security and Compliance:

• Implement and enforce data security measures to protect sensitive information.
• Ensure compliance with industry regulations and company policies related to data handling.
• Collaboration:
  • Collaborate with cross-functional teams, including data scientists, analysts, and software developers, to understand data requirements and deliver effective solutions.
  • Provide technical guidance and support to junior members of the data engineering team.

KNOWLEDGE AND SKILLS:

• Proficiency in Azure Data Factory, Azure SQL Database, Azure Databricks, and other relevant Azure services.
• Strong SQL skills and experience with data modeling techniques.
• Strong Python skills and experience.
• Demonstrated experience in designing and implementing scalable ETL processes.
• Familiarity with data integration and migration strategies.
• Solid understanding of data security and compliance best practices.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.

QUALIFICATIONS REQUIRED:

• Bachelor's degree in Computer Science, Information Technology, or a related field.

EXPERIENCE REQUIRED:

• Minimum of 5-10+ years of experience in data engineering, with a focus on Azure cloud technologies.

ADVANTAGEOUS:

• Azure certifications related to data engineering.
• Experience with big data technologies such as Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, MS Fabric or Apache Spark.
• Knowledge of data warehousing concepts and best practices.
• Experience with Power BI