This job offer is not available in your country.

BigData Platform Engineer / Architect

U3Singapore, Singapore

30+ days ago

Job description

Summary :

The Platform Engineer / Architect is responsible to define architecture and strategy for Batch workloads in Hybrid cloud environment. This includes POC design, enabling multitenancy and capacity planning at enterprise scale, component stress and load testing, deployment automation, monitoring and governance.

Responsibilities :

The main and routine tasks of this position are to :

Design and develop the Batch data processing on onprem or public cloud
Develop and maintain an advanced platform to manage Kubernetes clusters lifecycle for batch processing.
Understand the On prem VM based Bigdata ecosystem, designing and developing migration plan to on prem cloud native deployment.
Determine security and segregation of Lab and Factory / BAU workloads.
Develop tools for automated build, test, deployment and management of the platform.
Determine approach for multitenancy and capacity planning at enterprise scale.
Improve continuous integration and delivery systems.
Monitor system events to ensure health, maximum system availability and service quality
Maintain documentation regarding configurations, operations and troubleshooting procedures
Participate in the definition of standards, guidelines and best practices.
Assist in evaluating new requirements, technical design and standards.
Participate in R&D effort bringing new ideas and new technologies

Qualifications

A University Degree or equivalent experience in Computer Information Systems, Computer Science, Mathematics or a related field.

Relevant experience :

Architect : Have worked on Big Data for more than 8+ years and have hands on with Spark, Hive & Presto.

Developer : 2+ years of experience in Kubernetes / OpenShift.

Skills :

Good with Architecture and has good understanding and in depth knowledge on distributed systems

Professional experience building enterprise Big Data Applications using Spark, Hadoop, Hive etc.

Hands on Knowledge on Containerisation and deploying Opensource products on K8s / OpenShift.

Maintaining and building inhouse based Opensource platforms like Hadoop, Spark, Hive, Presto

Skill in developing techniques and methodologies to resolve unprecedented problems or situations

Ability to communicate effectively with all levels of management

Ability to communicate with remote teams

Ability to make complex information accessible

Must be a self-starter that requires only limited supervision / guidance

Candidate must present with honesty, integrity and maintains confidentiality at all times

Be a team player

Have good interpersonal communication skills

Ability to work effectively in a fast paced changing environment

Ability to multi-task and prioritize

Thorough experience in defining and applying design standards, patterns, quality matrix depending on the solutions

Deep knowledge of OOPs, data structure, and algorithm

Ability & willingness to learn technologies at pace

Knowledge :

Expertise with Java or any coding languages.

Linux Scripting (Bash, Python, etc.)

Expertise with Source Control Management (Git, Bitbucket, etc.)

Expertise with CI / CD (Jenkins / ArgoCD)

Expertise with Configuration Management (Ansible, Terraform etc.)

Expertise with Monitoring (Prometheus, etc.)

In-depth knowledge with Linux environments (RHEL)

Knowledge on Protegrity.

Knowledge of Cloud infrastructures (AWS EKS, Google GKS, Azure AKS, etc.)

Knowledge of Web Servers (Nginx, Apache, etc.)

Knowledge in RESTful API design and implementation

Knowledge in the development of high-performance and fault-tolerant systems

Knowledge in distributed systems programming

Experience with Java or Go programming language an asset

Experience with Docker an asset

Experience running Kubernetes in production an asset

Experience in large-scale production deployments an asset

Create a job alert for this search

Platform • Singapore, Singapore