Senior Data Engineer - D02 Anson, Tanjong Pagar, SG

Roles & Responsibilities

Responsibilities

• Integrate data from multiple sources, such as databases, APIs, or streaming platforms, to provide a unified view of the data
• Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of data
• Identify and resolve data quality issues, monitor data pipelines for errors, and implement data governance and data quality frameworks
• Enforce data security and compliance with relevant regulations and industry-specific standards
• Implement data access controls, encryption mechanisms, and monitor data privacy and security risks
• Optimise data processing and query performance by tuning database configurations, implementing indexing strategies, and leveraging distributed computing frameworks
• Optimize data structures for efficient querying and develop data dictionaries and metadata repositories
• Identify and resolve performance bottlenecks in data pipelines and systems
• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders
• Document data pipelines, data schemas, and system configurations, making it easier for others to understand and work with the data infrastructure
• Monitor data pipelines, databases, and data infrastructure for errors, performance issues, and system failures
• Set up monitoring tools, alerts, and logging mechanisms to proactively identify and resolve issues to ensure the availability and reliability of data
• It would be a plus if he has software engineering background

Requirements

Preferred Skills & Characteristics

Consistently display dynamic independent work habits, goal oriented, passionate in growth mindsets and self-motivated professional. Self-driven and proactive in keeping up with new technologies and programming

Tell employers what skills you have

Scala

Azure

Big Data

Data Modeling

Pipelines

Software Engineering

MySQL

Scripting

Data Integration

Data Quality

Data Engineering

SQL

Python

Java

Data Warehousing

Databases