Data Engineer - EC2 Capacity Data Analytics at Amazon

Job Duties

• Develop and maintain automated ETL pipelines for big data using languages such as Scala, Spark, SQL and AWS services such as S3, Glue, Lambda, SNS, SQS, KMS. Example: ETL jobs that process a continuous flow of JSON source files and output the data in a business-friendly Parquet format that can be efficiently queried via Redshift Spectrum using SQL to answer business question.
• Develop and maintain automated ETL monitoring and alarming solutions using Java/Python/Scala, Spark, SQL, and AWS services such as CloudWatch and Lambda.
• Implement and support reporting and analytics infrastructure for internal business customers using AWS, services such Athena, Redshift, Spectrum, EMR, and QuickSight.
• Develop and maintain data security and permissions solutions for enterprise scale data warehouse and data lake implementations including data encryption and database user access controls and logging.
• Develop and maintain data warehouse and data lake metadata, data catalog, and user documentation for internal business customers.
• Develop, test, and deploy code using internal software development toolsets. This includes the code for deploying infrastructure and solutions for secure data storage, ETL pipelines, data catalog, and data query.

Basic Qualifications

• Bachelor’s degree in Computer Science or related technical field, or equivalent work experience.
• 4+ years of overall work experience including Software Engineering, Data Engineering, Database Engineering, Business Intelligence.
• Experience with AWS technologies stack including Lambda, Glue, Redshift, RDS, S3, EMR or similar big data solutions stack

Preferred Qualifications

• Demonstrate efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
• Demonstrable proficiency in distributed systems and data architecture; design and implementation of batch and stream data processing pipelines; knows how to optimize the distribution, partitioning, and MPP of high-level data structures.

Amazon is an equal opportunities employer, and we value your passion to discover, invent, simplify and build. We welcome applications from all members of society irrespective of age, sex, disability, sexual orientation, race, religion or belief. Amazon is strongly committed to diversity within its community and especially welcomes applications from South African citizens who are members of designated groups who may contribute to Employment Equity within the workplace and the further diversification of ideas. In this regard, the relevant laws and principles associated with Employment Equity will be considered when appointing potential candidates. We are required by law to verify your ability to work lawfully in South Africa. Amazon requires that you submit a copy of either your identity document or your passport and any applicable work permit if you are a foreign national, along with an updated curriculum vitae.

Company - Amazon Dev Centre South Africa
Job ID: A1314605