ruby-labs - OpenSearch Engineer / AWS & Data Pipelines

• Own and operate AWS OpenSearch Service in production. • Design indices, mappings, analyzers, templates, aliases, shards, and retention policies. • Optimize search queries, aggregations, filtering, pagination, and bulk indexing. • Investigate slow queries, indexing bottlenecks, hot shards, memory pressure, and rejected requests. • Plan safe upgrades, migrations, reindexing, and capacity scaling. • Build and maintain reliable pipelines that synchronize PostgreSQL data with OpenSearch. • Support inserts, updates, deletes, backfills, and full reindexing. • Use AWS-native services such as AWS DMS, OpenSearch Ingestion, Lambda, SQS, S3, ECS, or EKS where appropriate. • Implement retries, idempotency, deduplication, replay, dead-letter handling, and failure recovery. • Monitor ingestion lag, throughput, failed records, and data freshness. • Build reconciliation processes to detect and repair missing, stale, or inconsistent documents. • Ensure ingestion does not negatively affect the transactional PostgreSQL workload. • Reliability and operations • Monitor cluster health, search latency, indexing latency, storage, CPU, JVM pressure, and shard distribution. • Build CloudWatch dashboards, alerts, and operational runbooks. • Design snapshot, restore, backup, and disaster-recovery procedures. • Participate in production incident response and root-cause analysis. • Implement secure access using IAM, role-based access control, encryption, and least-privilege principles. • Work closely with backend, database, infrastructure, and product teams. • Help design efficient OpenSearch document models for payments, customers, subscriptions, and transactions. • Define standards for searchable fields, schema changes, and index versioning. • Document architecture, data flows, and recovery procedures.