Data Engineer

Roles & Responsibilities

Key Responsibilities

• Collaborate across workstreams to support data requirements, including the development of reports and dashboards.
• Analyze and perform data profiling to identify patterns, discrepancies, and improvements in line with Data Quality and Data Management processes.
• Design and develop end-to-end (E2E) data pipelines : ingestion, transformation, processing, and surfacing of data for large-scale applications.
• Automate data pipelines using Azure, AWS data platforms, Databricks, and Data Factory .
• Translate business requirements into technical specifications for project design and delivery.
• Perform data ingestion in batch and real-time via file transfer, APIs, data streaming (Kafka, Spark Streaming).
• Develop ETL processes using Apache Spark to meet data transformation and standardization needs.
• Build data exports, APIs, and visualizations using Power BI, Tableau, or other visualization tools .
• Ensure alignment with best practices in data governance, security, and architecture .

Qualifications & Experience

Strong skills in :

Programming & Data Engineering : Python, SQL, Spark

Cloud & Data Platforms : Azure, AWS, Databricks, Data Factory

Architecture : Data / Solution Architecture, APIs

Tell employers what skills you have

Tableau

Azure

Big Data

Data Modeling

Pipelines

Data Transformation

Data Management

ETL

Data Quality

Data Governance

Data Engineering

SQL

Python

Visualization

Power BI

Data Warehousing