Senior Data Engineer at GVW Group

Job Title: Senior Data Engineer
Reports To: Functionally – Data Engineering Manager

Data Engineer Job Responsibilities:

• Serve as the subject matter expert for data and systems.
• Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.
• Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility and fostering data-driven decision making across the organization.
• Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
• Develop end-to-end ML pipelines encompassing the ML lifecycle from data ingestion, data transformation, model training, model validation, model serving, and model evaluation over time.
• Collaborate closely with AI scientists to accelerate deployments of ML algorithms to production.
• Setup CI/CD/CT pipelines, model repository for ML algorithms
• Deploy models as a service
• Contributes to engineering wiki, and documents work.
• Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
• Works closely with cross-functional teams of frontend and backend engineers, product managers, and analyst to enhance data models and support advanced BI and analytics.
• Defines company data assets (data models), ETL jobs to populate data models.
• Designs data integrations and data quality framework.
• Designs and evaluates open source and vendor tools for data lineage.
• Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.
• Mentor junior data engineers, lead code reviews, and promote best practices and skill development.

Data Engineer Qualifications / Skills:

• Knowledge of best practices and IT operations in an always-up, always-available service
• Establish and promote best practices for data pipeline and model development.
• Experience with or knowledge of Agile Software Development methodologies
• Excellent problem solving, creativity, attention to detail and troubleshooting skills
• Process oriented with great documentation skills
• Excellent oral and written communication skills with a keen sense of customer service
• Eagerness to learn and upskill to stay at the forefront of technology offerings in the market.
• Understanding of ML Algorithms, experience creating & executing efficient MLOps pipelines, and tuning ML models
• Team player mindset with an enthusiasm for collaboration.

Education, Experience, and Licensing Requirements:

Must Have:

• BSc or MSc degree in Computer Science or a related technical field
• 5+ years of Python or R development experience
• 5+ years of MS SQL experience (PostgreSQL experience is a plus)
• 5+ years of experience with Warehouse Architecture, schema design and dimensional data modeling.
• 5+ years of experience in Data Analytics and Business Intelligence tools such as Power BI
• Ability in managing and communicating data warehouse plans to internal clients
• Experience designing, building, and maintaining data processing systems on multiple platforms both Cloud (Azure, AWS, MS Fabric is a plus) and On-Premises (MS SQL Server, SSIS)
• Experience in ML Model deployment, ML frameworks and libraries
• Good experience in Apache Spark.
• Experience debugging and reasoning about production issues is desirable.
• Experience presenting demos and training of technical, non-technical and analytical resources

Advantages to have:

• Experience in data streaming is advantageous i.e. Kafka and/or AWS Kinesis
• IoT device and systems integration