Senior Data Engineer

Roles & Responsibilities

Role Overview

The Senior Data Engineer is responsible for designing, building, and maintaining large-scale, secure, and high-performance data pipelines supporting critical Financial Services workloads.

The role focuses on data modernization, regulatory data aggregation, and AI / ML enablement across domains such as Core Banking, Payments, Risk, Treasury, and Regulatory Reporting.

2. Key Responsibilities

• Design, implement, and optimize ETL / ELT data pipelines using Apache Spark, PySpark, Databricks, or Azure Synapse.
• Build and operationalize real-time streaming pipelines leveraging Kafka / Confluent / Azure Event Hubs for risk and liquidity data.
• Integrate and transform data across Core Banking, Trade, Payments, Treasury, CRM, and Compliance systems.
• Implement data quality, validation, and lineage controls using tools such as Great Expectations / Deequ / dbt tests.
• Develop and maintain data models and schemas (3NF, Dimensional, Data Vault 2.0).
• Collaborate with Security and Governance teams to implement data security, masking, encryption, and tokenization in compliance with MAS TRM / PDPA / PCI-DSS.
• Participate in data platform modernization projects (Teradata / DB2 → Snowflake / Databricks / Synapse).
• Collaborate with Data Scientists and AI Engineers to deploy ML feature stores and model-serving pipelines.
• Support regulatory reporting (MAS 610 / 649) and Basel III / IV data flows.
• Maintain CI / CD pipelines for data infrastructure using Azure DevOps / Terraform / GitHub Actions.

3. Required Technical Skills

4. Experience and Qualifications

5. Key Attributes

Tell employers what skills you have

Microsoft Azure

PySpark

Factory

Airflow

Apache Spark

Scala

Azure

Pipelines

Data Quality

Data Engineering

Logging

SQL

Python

Tokenization

Orchestration

Apache