The action=wbsearchentities API currently gets continue+limit results from the underlying search (where continue effectively functions as an offset), converts all of them into API results (including an extra page ID lookup), then slices the first continue away. We should do this slicing before converting the search results into API results, thus potentially skipping a lot of page ID lookups.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Security | Lucas_Werkmeister_WMDE | T352877 Several efficiency errors in wbsearchentities API compound to long execution time and many database queries | ||
| Resolved | Lucas_Werkmeister_WMDE | T355053 Only create needed search result entries in wbsearchentities |
Event Timeline
Comment Actions
Change 990638 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Only build result entries for used wbsearchentities results
Comment Actions
Change 990638 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Only build result entries for used wbsearchentities results
Comment Actions
Change 991059 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@wmf/1.42.0-wmf.14] Only build result entries for used wbsearchentities results
Comment Actions
Change 991060 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@wmf/1.42.0-wmf.13] Only build result entries for used wbsearchentities results
Comment Actions
Change 991328 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@REL1_41] Only build result entries for used wbsearchentities results
Comment Actions
Change 991328 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@REL1_41] Only build result entries for used wbsearchentities results
Comment Actions
Change 991059 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.42.0-wmf.14] Only build result entries for used wbsearchentities results
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-01-17T14:16:20Z] <logmsgbot> lucaswerkmeister-wmde@deploy2002 Started scap: Backport for [[gerrit:991059|Only build result entries for used wbsearchentities results (T355053)]]
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-01-17T14:17:44Z] <logmsgbot> lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde: Backport for [[gerrit:991059|Only build result entries for used wbsearchentities results (T355053)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-01-17T14:25:44Z] <logmsgbot> lucaswerkmeister-wmde@deploy2002 Finished scap: Backport for [[gerrit:991059|Only build result entries for used wbsearchentities results (T355053)]] (duration: 09m 23s)
Comment Actions
Change 991330 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@REL1_40] Only build result entries for used wbsearchentities results
Comment Actions
Change 991060 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.42.0-wmf.13] Only build result entries for used wbsearchentities results
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-01-17T14:43:11Z] <logmsgbot> lucaswerkmeister-wmde@deploy2002 Started scap: Backport for [[gerrit:991062|Skip tainted references test:distnodiff script to fix Wikibase CI (T354881)]], [[gerrit:991060|Only build result entries for used wbsearchentities results (T355053)]]
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-01-17T14:44:39Z] <logmsgbot> lucaswerkmeister-wmde@deploy2002 lucaswerkmeister-wmde: Backport for [[gerrit:991062|Skip tainted references test:distnodiff script to fix Wikibase CI (T354881)]], [[gerrit:991060|Only build result entries for used wbsearchentities results (T355053)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2024-01-17T14:51:40Z] <logmsgbot> lucaswerkmeister-wmde@deploy2002 Finished scap: Backport for [[gerrit:991062|Skip tainted references test:distnodiff script to fix Wikibase CI (T354881)]], [[gerrit:991060|Only build result entries for used wbsearchentities results (T355053)]] (duration: 08m 28s)
Comment Actions
Change 991330 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@REL1_40] Only build result entries for used wbsearchentities results
Comment Actions
Change 991338 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@REL1_39] Only build result entries for used wbsearchentities results
Comment Actions
Change 991338 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@REL1_39] Only build result entries for used wbsearchentities results