Senior Data Engineer( Hive SQL, Hadoop, Cloudera)

Roles & Responsibilities

Responsibilities

• Migrate existing data warehouse and data model from MariaDB to Hadoop Big Data Cloudera on-premises platform.
• Develop and optimize SQL, Hive SQL, and Spark scripts to ensure efficient data processing.
• Design and implement data models to support business requirements and optimize performance.
• Collaborate with cross-functional teams to understand data requirements and ensure data integrity throughout the migration process.
• Develop and execute test plans to validate data accuracy and system performance.
• Coordinate with stakeholders (internal) to plan and execute production deployments.
• Schedule and monitor jobs using Autosys to ensure timely execution and minimize downtime.
• Provide technical expertise and guidance to team members throughout the migration project.
• Document processes, procedures, technical specifications, and best practices to facilitate knowledge sharing and ensure scalability.
• Create and maintain unit test case documents to ensure code quality and reliability.

Requirements :

Tell employers what skills you have

Version Control

Scalability

Big Data

Data Modeling

Interpersonal Skills

Hadoop

ETL

MariaDB

Reliability

Data Engineering

SQL

Autosys

Scheduling

Business Requirements