Senior Data Engineer at Juru Holdings

Role Overview

• Senior Data Engineer will lead the development and optimization of our data infrastructure, playing a critical role in shaping the future of our data architecture.
• They will be responsible for designing and implementing complex data solutions, mentoring team members, and collaborating with stakeholders to drive innovative data initiatives.
• This position requires extensive experience in data engineering, a deep understanding of advanced data technologies, and the ability to provide strategic guidance.

Key Responsibilities

• Architectural Leadership: Lead the design and implementation of scalable, high-performance data architectures.
• Provide strategic input into the evolution of the company's data infrastructure and technology stack.
• Advanced Data Modelling: Design and implement advanced data models to meet complex business requirements. Optimize data storage and retrieval mechanisms for optimal performance.
• Big Data and Advanced Analytics: Champion the use of advanced analytics and big data technologies to derive valuable insights.
• Work on advanced analytics solutions, including machine learning pipelines and predictive modelling.
• Performance Optimization: Oversee the optimization of data pipelines, databases, and ETL processes for maximum efficiency. Conduct in-depth performance tuning and troubleshooting of complex data workflows.
• Team Leadership and Mentorship: Lead a team of data engineers, providing technical guidance and mentorship. Foster a culture of continuous learning and collaboration within the data engineering team.
• Cross-functional Collaboration: Collaborate with data scientists, analysts, software developers, and other stakeholders to understand business needs and deliver comprehensive data solutions.
• Act as a key liaison between the data engineering team and other business units.
• Data Governance and Compliance: Establish and enforce data governance policies, ensuring data quality, integrity, and compliance with regulatory requirements.
• Stay abreast of industry best practices and emerging trends in data governance.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• 10+ years of experience in data engineering with a focus on complex data solutions.
• Expert proficiency in programming languages such as Python, SQL, and advanced scripting languages.
• In-depth knowledge of data warehousing concepts, big data technologies, and cloud platforms (e.g., AWS, Azure, Google Cloud).
• Proven experience in leading and mentoring a team of data engineers.
• Strong analytical and problem-solving skills with a strategic mindset.
• Excellent communication and collaboration skills.

Preferred Skills

• Certifications in relevant data engineering, cloud, or advanced analytics technologies.
• Experience with real-time data processing and streaming technologies.
• Knowledge of containerization and orchestration tools (e.g., Docker, Kubernetes).
• Familiarity with data security and privacy considerations.

Tech Stack

Core Tech Skills

• SQL
• Python
• Spark (PySpark & SQL)
• Hadoop

Specific Tech Skills

• AWS S3
• AWS Lambda
• AWS Redshift (Serverless, including Redshift Spectrum)
• AWS EMR
• AWS Step Functions
• AWS EventBridge Rules & Pipes
• AWS Glue
• AWS SQS
• AWS SNS
• AWS API Gateway
• AWS Athena
• AWS VPC
• AWS KMS
• AWS Secrets Manager
• io tables (Open table format)
• Unity Catalog (Open Catalog of Tables, Views, Functions, Models)
• Dynamo DB

Developer/Productivity Tools

• AWS Cloud Development Kit (CDK)
• Docker (for Dev Containers)
• Azure DevOps
• Xray
• JIRA
• Confluence (Wiki)
• io