Page MenuHomePhabricator

Create high level plan for the migration of the Elasticsearch 7.10 search cluster to replacement backend search engine
Open, MediumPublic3 Estimated Story Points

Description

Before starting work, we want to have a high level understanding of what needs to happen, identify potential problematic points and dependencies on other teams (if they exist).

AC

  • List of phab task for the main pieces of the migration
  • License inquiry on Elasticsearch 8
  • Brief on discussion with LTR plugin maintainers
  • SPIKE on vector embeddings of OpenSearch
  • Community Resources communication plan, including for as many parts of the Wikimedia software ecosystem as possible (including addition to Asana calendar(s))
  • Cloud Elastic - check logs for usage
  • Cloud Elastic - communicate timing
  • Cloud Elastic - migrate
  • Cloud Elastic - add vector based search support (may require standalone instances)
  • Dev developer environments (MediaWiki-Docker with Cindy, maybe another environment such as Catalyst if it's stable; probably not MW QuickStart based on discussions)
  • Product Analytics consultation
  • Search metrics
  • Intervention analysis of upgrades along the way to verify no issues (e.g., HTTP headers that make their way into refined tables)
  • Cindy the browser bot built for replacement
  • Review, report of any analysis chain changes
  • Analysis chain change fixes
  • Confirm Toolhub compatibility
  • Review breaking changes
  • Migrate plugins
  • Migrate Relforge
  • Deprecation logging
  • Deprecation and breaking changes fixes
  • Migration guide (or tooling, or both; TBD) for third party CirrusSearch installations
  • Update Extension:Translate
  • apifeatureusage
  • Translatewiki.net - align schedule for TWN (probably with Federico)
  • deployment-prep / beta
  • Alternate cluster - migrate
  • Main cluster - migrate

Event Timeline

Gehel triaged this task as Medium priority.Jul 16 2024, 12:53 PM
Gehel created this task.
Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.
Gehel set the point value for this task to 3.Jul 22 2024, 3:32 PM
dr0ptp4kt renamed this task from Create high level plan for the migration of the Elasticsearch search cluster to OpenSearch to Create high level plan for the migration of the Elasticsearch Elasticsearch 7.10 search cluster to replacement backend search engine.Sep 9 2024, 8:06 PM
dr0ptp4kt updated the task description. (Show Details)
dr0ptp4kt renamed this task from Create high level plan for the migration of the Elasticsearch Elasticsearch 7.10 search cluster to replacement backend search engine to Create high level plan for the migration of the Elasticsearch 7.10 search cluster to replacement backend search engine.Sep 10 2024, 1:19 AM