Nestjs search

Search Engine & Full-Text

Priority: P1 (OPERATIONAL)

Full-text search implementation with Elasticsearch and database patterns.

• Pattern: CQRS (Command Query Responsibility Segregation).
  • Write: To Primary Database (Postgres/MySQL). Source of Truth.
  • Read (Complex): To Search Engine (Elasticsearch, OpenSearch, MeiliSearch). Optimized for filtering, fuzzy search, and aggregation.

Synchronization (The Hard Part)

• Dual Write (Anti-Pattern): await db.save(); await es.index();.
  • Why: Partial failures leave data inconsistent. Slows down HTTP response.
• Event-Driven (Recommended):
  1. Service writes to DB.
  2. Service emits EntityUpdated.
  3. Event Handler (Async) pushes to Queue (BullMQ).
  4. Worker indexes document to Search Engine with retries.
• CDC (Golden Standard): Change Data Capture (Debezium). Connects directly to DB transaction log. No app conceptual overhead, but higher ops complexity.

Organization

• Module: Encapsulate the client in a SearchModule.
• Abstraction: Create generic SearchService<T> helpers.
  • indexDocument(id, body)
  • search(query, filters)
• Mapping: Use class-transformer to map Entities to "Search Documents". Search docs should be flatter than Relational entities constraints.

Testing

• E2E: Do not mock the search engine in critical E2E flows.
• Docker: Spin up elasticsearch:8 container in the test harness to verify indexing works.

Anti-Patterns

• No dual writes to DB + ES: Use event-driven or CDC pattern; dual writes risk partial failure inconsistency.
• No Elasticsearch for structured queries: Use DB indexes for filtering; ES is for full-text and complex search.
• No ES mocks in E2E search tests: Spin up elasticsearch:8 container to verify indexing behavior accurately.