Roles & Responsibilities
About the Role :
We are seeking a skilled and motivated Databricks Engineer to lead the migration of on-premises data warehouses to Databricks. This role is crucial for designing and maintaining a robust, integrated, and governed data infrastructure while ensuring high performance, security, and compliance.
Key Responsibilities :
- Lead end-to-end data migration projects from on-premises environments to Databricks with minimal downtime.
- Collaborate with architects to design solutions that meet functional and non-functional requirements.
- Implement solutions on Databricks with hands-on experience in Amazon Web Services (AWS) environments.
- Configure Databricks clusters, write PySpark code, and build CI / CD pipelines for deployments.
- Apply optimization techniques such as Z-ordering, auto-compaction, and vacuuming.
- Process near real-time data using Auto Loader and DLT pipelines.
- Identify, communicate, and mitigate risks and issues related to data processes.
- Ensure data availability and integrity while resolving data-related issues.
- Optimize AWS and Databricks resource usage to control costs while maintaining performance and scalability.
- Stay updated with the latest Databricks and AWS advancements and data engineering best practices.
- Implement engineering methodologies, standards, and leading practices proactively.
Requirements :
Bachelor's Degree in Computer Science, Data Engineering, or a related field.At least 5 years of experience in data engineering, with expertise in AWS or Azure services, Databricks, and / or Informatica IDMC.Proficiency in Python, Java, or Scala for building data pipelines.Excellent hands-on knowledge of SQL and NoSQL databases.Experience in data modelling, schema design, and performance optimization for complex data transformations.Familiarity with containerization and orchestration tools like Docker and Kubernetes.Excellent problem-solving, analytical, and communication skills.Interested candidates may send in their resume and cover letter directly to gem.cabria@manpower.com.sg (R1434374), stating the position as the subject title in the email.
Jireli Gem Mejia Cabria EA License No. : 02C3423 Personnel Registration No. : R1434374
Please note that your response to this advertisement and communications with us pursuant to this advertisement will constitute informed consent to the collection, use and / or disclosure of personal data by ManpowerGroup Singapore for the purpose of carrying out its business, in compliance with the relevant provisions of the Personal Data Protection Act 2012. To learn more about ManpowerGroup's Global Privacy Policy, please visit https : / / www.manpower.com.sg / privacy-policy
Tell employers what skills you have
PySpark
Scalability
Scala
Kubernetes
Azure
Pipelines
Informatica
Architects
Amazon Web Services
Data Engineering
Data Migration
Containerization
Docker
Orchestration
Databases