Big Data Engineer

Roles & Responsibilities

Key Responsibilities

• Lead the design and implementation of a high-performance, real-time data platform for financial systems, supporting petabyte-scale daily transaction data across ingestion, storage, and computation layers.
• Architect a cloud-native data warehouse solution on AWS, incorporating Spark and Flink to support both batch and streaming use cases with millisecond-level latency for payment data insights.
• Partner with key business units (e.g., acquiring and issuing teams) to develop data masking and auditing mechanisms that comply with PCI DSS and other regulatory standards.
• Design and maintain dashboards that enable data-driven decision-making across business functions.
• Integrate data from diverse payment systems to enable consistent, reliable, and scalable data flow across platforms.
• Own the full data pipeline—from ingestion and processing to modeling and standardization—ensuring high data quality, consistency, and accessibility.
• Enable cross-functional teams by driving data standardization, automation, and seamless access to clean, actionable datasets.
• Collaborate with compliance, risk, and product teams to deliver secure and reliable data solutions in a regulated environment.

Required Qualifications

Preferred Qualifications

Tell employers what skills you have

Modeling

Big Data

Data Modeling

Data Structures

Architect

Hadoop

ETL

Data Quality

Auditing

Data Governance

Tuning

Data Engineering

PCI DSS

Compliance

Communication Skills

Accessibility