Senior Data Engineer - Countrywide at DataOrbis

Responsibilities:

Data Engineering

• Gather and cleanse data from a variety of sources, including structured and unstructured data
• Design, develop, and maintain GCP-based data infrastructure, including data pipelines and ETL processes using Google BigQuery and DBT
• Implement data transformations, aggregations, and data cleansing to support analytics and reporting

Data Modeling

• Utilize DBT (Data Build Tool) to define, document, and maintain data models and business logic in SQL for analytics and reporting
• Develop and implement data quality checks to ensure the accuracy and integrity of data

Performance Optimization

• Monitor and optimize data pipelines to ensure they are performing as expected, ensuring efficient data processing and cost management
• Automate data processing tasks to improve efficiency and scalability

Data Governance

• Enforce data quality standards, implement data access controls, and ensure compliance with data privacy regulations

Visualization

• Analyze and interpret data to present insights clearly and concisely
• Prepare dashboard visualizations and reports to communicate data insights to internal and external stakeholders in written and/or verbal format

Machine Learning Integration

• Work with data scientists and analysts to build and deploy data-driven solutions, using Google Vertex AI

Team Collaboration

• Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand their data needs and provide solutions
• Lead and mentor other data engineers to design and implement complex data solutions

Documentation

• Create and maintain documentation for data pipelines, data models, and best practices

Technical Innovation

• Stay up-to-date on the latest GCP and data engineering technologies to incorporate them into the data infrastructure

Requirements

Qualifications

• Minimum bachelor’s degree in computer science, data, engineering data science, or a related field
• Post-graduate degree in any of the above will be an advantage
• 5+ years of experience in Google Cloud Data Engineering or a comparable cloud platform is preferred
• In-depth knowledge of GCP services, including Google BigQuery, SQL and Cloud Storage would be an added advantage.
• Experience with data modeling and data warehousing principles

Preferred Skills

• Knowledge of DBT, or similar technology, and experience in building and maintaining data models
• Familiarity with Google Vertex AI for ML integration
• Experience with data governance and compliance practices
• Google Cloud certifications such as Google Cloud Professional Data Engineer are a significant advantage
• Excellent problem-solving and analytical skills with the ability to troubleshoot data-related issues
• Strong communication and teamwork skills
• Ability to work independently and as part of a team
• Ability to work under pressure and meet deadlines