> ## Documentation Index
> Fetch the complete documentation index at: https://ihub.learnille.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Elasticsearch to OpenSearch Migration Plan

> Migrated from the previous Learnille internal documentation hub

## Purpose

Document the feasibility and plan for migrating from Elasticsearch 8.x to OpenSearch 2.x. This migration would reduce licensing concerns and costs while maintaining feature parity.

## Current State

### Elasticsearch Usage in Codebase

| Feature | How We Use It | Files |
| - | - | - |
| **Full-text search** | Course/consultation search | `search.service.ts` |
| **KNN/Vector search** | Product embeddings, user embeddings | `recommendation.service.ts`, `user-embedding.service.ts` |
| **Painless scripts** | Scoring, field calculations | Various query builders |
| **PIT pagination** | Deep pagination for large result sets | `search.service.ts` |
| **Aggregations** | Faceted search, analytics | `search.service.ts` |

### Current Client

* **Package**: `@elastic/elasticsearch` v8.x
* **Connection**: Via `ElasticsearchService` from `@nestjs/elasticsearch`

## Compatibility Assessment

### What Works the Same

| Feature | Compatibility | Notes |
| - | - | - |
| Basic CRUD | Full | Index, get, delete, bulk operations identical |
| Full-text queries | Full | match, multi\_match, bool queries unchanged |
| Aggregations | Full | Terms, range, nested aggs work the same |
| PIT pagination | Full | Point-in-time API available in OpenSearch |
| Painless scripts | High | Core Painless syntax compatible |

### What Requires Changes

| Feature | Change Required | Effort |
| - | - | - |
| **Client library** | `@elastic/elasticsearch` → `@opensearch-project/opensearch` | Low |
| **Vector field type** | `dense_vector` → `knn_vector` | Medium |
| **KNN query syntax** | Different query structure | Medium |
| **Index mappings** | Update vector field definitions | Low |

## Migration Steps

### Phase 1: Client Swap (Day 1)

1. **Install OpenSearch client**
   ```bash theme={null}
   npm uninstall @elastic/elasticsearch @nestjs/elasticsearch
   npm install @opensearch-project/opensearch
   ```

2. **Create OpenSearch module wrapper**
   * Create `opensearch.module.ts` to replace `ElasticsearchModule`
   * Maintain same service interface

3. **Update imports** across codebase
   * Find/replace `ElasticsearchService` references

### Phase 2: Vector Search Migration (Day 1-2)

1. **Update index mappings** for vector fields

   **Before (Elasticsearch):**

   ```json theme={null}
   {
     "product_embedding": {
       "type": "dense_vector",
       "dims": 1536,
       "index": true,
       "similarity": "cosine"
     }
   }
   ```

   **After (OpenSearch):**

   ```json theme={null}
   {
     "product_embedding": {
       "type": "knn_vector",
       "dimension": 1536,
       "method": {
         "name": "hnsw",
         "space_type": "cosinesimil",
         "engine": "nmslib"
       }
     }
   }
   ```

2. **Update KNN query syntax**

   **Before (Elasticsearch 8.x):**

   ```typescript theme={null}
   {
     knn: {
       field: 'product_embedding',
       query_vector: userEmbedding,
       k: 10,
       num_candidates: 50
     }
   }
   ```

   **After (OpenSearch):**

   ```typescript theme={null}
   {
     query: {
       knn: {
         product_embedding: {
           vector: userEmbedding,
           k: 10
         }
       }
     }
   }
   ```

3. **Reindex data** with new mappings

### Phase 3: Testing & Validation (Day 2-3)

1. **Unit tests** - Verify all search functions work
2. **Integration tests** - Test against OpenSearch instance
3. **Performance comparison** - Benchmark KNN search latency
4. **Data validation** - Ensure search results quality unchanged

### Phase 4: Deployment

1. Set up OpenSearch cluster (or use managed service)
2. Migrate data from Elasticsearch
3. Update environment configurations
4. Deploy updated application
5. Monitor for issues

## Files to Modify

| File | Changes |
| - | - |
| `server/src/elastic/elastic.module.ts` | Replace with OpenSearch module |
| `server/src/elastic/elastic.service.ts` | Update client usage |
| `server/src/elastic/indexes/*.ts` | Update vector field mappings |
| `server/src/recommendation/services/*.ts` | Update KNN query syntax |
| `server/src/search/search.service.ts` | Update query builders |
| `package.json` | Swap client packages |

## Estimated Effort

| Phase | Time |
| - | - |
| Client swap | 0.5 day |
| Vector search migration | 1 day |
| Testing & validation | 1 day |
| **Total** | **2-3 days** |

## Risks & Mitigations

| Risk | Mitigation |
| - | - |
| Query syntax differences we missed | Comprehensive integration tests |
| Performance regression on KNN | Benchmark before/after, tune HNSW parameters |
| Painless script edge cases | Review all script\_score queries |
| Data migration issues | Run parallel environments during transition |

## Decision

**Status:** Planned for future (not urgent)

**Rationale:** Current Elasticsearch setup works well. Migration provides cost savings but requires dedicated effort. Plan preserved here for when the time is right.

## Resources

* [OpenSearch Documentation](https://opensearch.org/docs/latest/)
* [OpenSearch KNN Plugin](https://opensearch.org/docs/latest/search-plugins/knn/index/)
* [ES to OpenSearch Migration Guide](https://opensearch.org/docs/latest/upgrade-to/index/)

## Last Updated

2025-01-27


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.