Principal Data Engineer at Nintex

About the role: 

• You are an expert in data acquisition and processing not only in your team’s domain, but also across Nintex. Your technical leadership demonstrates a passion for quality data engineering and is demonstrated across multiple, complex projects and in assistance to other teams. You mentor other engineers and provide experienced advice in your specialties across other teams. You look for opportunities to mine data from development and operations that usefully informs development and product decisions. Your data engineering pipelines are designed with the possible downstream requirements of data analysts and scientists in mind. 
• As a Principal Data Engineer, you are focused on the work of your team and domain, but you also have a larger role across Nintex and in the outside data engineering community. You will participate in larger projects on occasion, but your focus is designing, developing, implementing and mentoring on your team’s assigned deliverables. 

Your contribution will be:  

• You ensure that data history is sufficient for data science modelling requirements.
• You identify and forward Intellectual Property (IP) opportunities to the Nintex Patent process.
• You coordinate the data inputs from multiple sources to fashion an end-to-end pipeline to create report artifacts that meet all reporting objectives of management stakeholders.
• You create and maintain instrumentation that surfaces data ingestion systems’ stability and availability metrics.
• You use Azure event hubs and functions to implement required data operations while leveraging Azure product strengths and compensating for its limitations (scaling, error handling, and etc.).
• You apply different data pipeline architectures appropriately to meet product requirements (data warehouse, data lake, data lakehouse, etc.).
• You identify and implement opportunities to optimize ETL processing to efficiently process large data sets.
• You develop or adapt partitioning and data organization techniques for optimal querying of big data.
• You implement data storage, access structures, and security protocols that meet Nintex’s data sovereignty and Personally Identifying Information (PII) policies.
• You are an active part of the incident management process, including on-call rotation and being a key resource in the escalation response.
• You work with data providers to document the incoming data stream’s domain boundaries, datatypes, and term definitions in sufficient detail to enable transformation of the data for end-user consumption. 

To be successful, we think you need:  

• You have 8+ years experience in data engineering
• Hands-on experience with PaaS
• Experience with DataBricks, Azure, Unity Catalog
• Experience with Hadoop/Spark, Kafka and Azure OpenAI
• Experience with Scala and Python is advantageous
• Experience with real-time data streaming and any distributed computing experience would be advantageous