Observability Engineer

Observability Engineer   -   ( 24000139 )

Description

The engineer is responsible for designing, implementing, and maintaining Elasticsearch clusters and related components in enterprise distributed cluster environment. His role involves working with large-scale data sets and optimizing search and indexing performance.

Responsibilities

• Cluster Design and Deployment : Designing and deploying Elasticsearch clusters based on specific requirements, including hardware, network, and security considerations.
• Configuration and Optimization : Configuring Elasticsearch settings, including index mappings, shard allocation, and cluster settings, to optimize performance and ensure efficient resource utilization.
• Data Ingestion and Indexing : Developing strategies for efficient data ingestion into Elasticsearch, including handling large data volumes, implementing data pipelines, and optimizing indexing processes.
• Search and Query Optimization : Tuning search queries, relevance scoring, and aggregations to improve search performance and response times.
• Monitoring and Troubleshooting : Implementing monitoring solutions to track cluster health, performance, and resource utilization. Identifying and resolving issues related to indexing, search, and cluster stability.
• Scalability and High Availability : Planning and implementing strategies for scaling Elasticsearch clusters horizontally or vertically to handle increased data volumes and user traffic. Ensuring high availability and fault tolerance through replication and data redundancy.
• Security and Access Control : Implementing security measures, such as authentication, authorization, and encryption, to protect Elasticsearch clusters and data from unauthorized access.
• Collaboration and Documentation : Collaborating with cross-functional teams, including developers, DevOps, and data engineers, to integrate Elasticsearch into applications and systems. Documenting cluster configurations, deployment processes, and troubleshooting guidelines.
• Performance Analysis and Optimization : Conducting performance analysis, identifying bottlenecks, and implementing optimizations to improve overall system performance and efficiency.
• Stay updated with Elasticsearch : Keeping up-to-date with the latest features, enhancements, and best practices in Elasticsearch and related technologies.
• Collaborate and work with the various stakeholder to develop technical solutions, plans and configuration for the observability capability and develop run book automation with the objective to achieve self-healing capability.
• Perform integration and develop full interoperability capabilities with various operations management platform including change management, service management, privileged access management system, etc.
• Provide support to existing monitoring system and bridge the transition to an AI-enabled observability platform.

The engineer plays a crucial role in ensuring the smooth operation and performance of Elasticsearch clusters, enabling efficient search and analysis capabilities for organizations dealing with large-scale data sets.

Qualifications

Requirements

Primary Location

:   Singapore

Job

:   Technology Solutions

Organization

:   Group Operations & Technology Division

Schedule

:   Permanent

:   Full-time

Job Posting

:   08-Sep-2024, 3 : 49 : 52 AM