Change Details

[[ https://www.mediawiki.org/wiki/Topic:Ue4bfpalv9hi25w5 | A user reports very slow load times ]] for Watchlist. The user has a large Watchlist of over 5500 pages, but his load time with filters of 27 seconds seems too long. Trizek, for example, has 4000 pages on his Watchlist, and it loads in just over 3 seconds. The user has provided lots of detail about his filters and configuration. (I've asked him to try the load in safemode and report back.) One reason that I'd like to look into this is that I've noticed that times for the new filters MAY have gotten slower over the last 6 months. See the stats below, which compare times averaged over a week from recently and six months prior (you can click on the dates to check out for yourself). | **When** | **RC Page avg ready** | **Watchlist avg ready** | | [[ https://grafana.wikimedia.org/dashboard/db/rcfilters-performance?orgId=1&from=1526828440288&to=1527357640288&var-interval=10m | May 20-26 ]] | 2.9 secs | 3.88 secs | [[ https://grafana.wikimedia.org/dashboard/db/rcfilters-performance?orgId=1&from=1511712040288&to=1512241240288&var-interval=10m | Nov 26-Dec 2 ]] | 2.5 secs | 3.4 secs | --- ## Summary of issues These are sourced from [Plans to graduate the New Filters on Watchlist out of beta on Talk:Edit Review Improvements/New filters for edit review](https://www.mediawiki.org/w/index.php?title=Topic:Ue4bfpalv9hi25w5&topic_showPostId=ufhj7291s5boc651&fromnotif=1#flow-post-ufhj7291s5boc651) and [Slow and unresponsive on Talk:Edit Review Improvements/New filters for edit review](https://www.mediawiki.org/w/index.php?title=Topic:Tztj44ddxkip6avl&topic_showPostId=ufhd3edibkh1bomd&fromnotif=1#flow-post-ufhd3edibkh1bomd). Thanks also to [Amorymeltzer](https://www.mediawiki.org/wiki/User:Amorymeltzer) who generously took the time to generate a JS profile and shared it with us. The highlights: 1. Some users, but not all, are unable to click on any links on the page while the RC Filters JS loads. This is probably a CSS issue but we need to investigate further. 2. All the bug reports are dealing with larger watchlists and also larger `limit` properties when viewing their watchlists. All the users who submitted feedback are using `limit=500` or `limit=1000`. - Bringing the `limit` down to 100 items seems to bring users back into the range of acceptable performance, but these users are not satisfied with `limit=100`, they need to be able to work with 500-1000 items at a time. 3. The bugs have been reported for the watchlist but [at least one user](https://www.mediawiki.org/w/index.php?title=Topic:Ue4bfpalv9hi25w5&topic_showPostId=ufhi2cdoyp4ptci9#flow-post-ufhi2cdoyp4ptci9) sees similar performance problems on `Special:RecentChanges` 4. The initial request time to get data from the server for the watchlist is slow, it takes at least 4-5 seconds per page request. That means at a minimum each individual filter toggle will take 4-5 seconds. - Testing locally, on a watchlist with 25 items, I have a DOMContentLoaded time of 4.66 seconds, and page load at 30.71s. Toggling filters is between 0.6 and 1.5 seconds. Testing with another account locally with a watchlist of 2500 items, and setting limit to 1000, DOMContentLoaded is 42.77 seconds, and page load is 55.53. Toggling filters on a watchlist of this size takes about 12-13 seconds. - On the watchlist with 2500 items, if I set the limit to 25, the DOMContentLoaded time is 4.73s and the page load is 10.02s, toggling filters takes 2-3 seconds. 5. The initial click on the filters box requires an additional call to load the JS/CSS needed to show the expanded filters interface. In addition to taking a few seconds, this seems to also block user interaction with the page. - We could consider loading this data asynchronously once the initial watchlist has loaded. It’s not a lot of data to add to the page by default. 5. Once the filter UI is open, looking at the JS profiler for toggling the checkboxes on the filters, `setVisibleItems()` (in `mw.rcfilters.dm.FilterGroup.js`) and `setupHighlightContainers()` (in `mw.rcfilters.ui.ChangesListWrapperWidget.js`) pop up as slower functions. We need to do some more investigation of the JS flamegraph to see what other functions are slow and what if anything we can optimize. 6. Our bug report sample size is pretty small. We have ~70k users with this feature enabled but only have a handful of bug reports. It's possible that this issue is more widespread and we haven't seen the feedback, or it's possible that it's limited to users with both larger watchlists and larger `limit` properties set in their query. ## Send us your performance traces If you'd like to help us troubleshoot performance issues on your Watchlist, please copy the text below and paste it into a new comment on this task, and fill in each section. - **URL:** //e.g. https://en.wikipedia.org/wiki/Special:Watchlist?hidemyself=1&hidebots=1&hidecategorization=1&hideWikibase=1&limit=500&days=7&enhanced=1&damaging__likelybad_color=c4&damaging__verylikelybad_color=c5&goodfaith__maybebad_color=c3&urlversion=2&safemode=1// - **Browser and version:** //(e.g. Firefox 60)// - **OS:** //e.g. Ubuntu 18.04// - **CPU:** //e.g. i5 dual-core// - **RAM:** //e.g. 8 GB// - **Number of items in watchlist:** //e.g. 5,000// - **Observations:** //Please include any comments or observations you have.// ### Performance trace: Please ensure that `&safemode=1` is added to your URL before doing the performance trace. **Chrome(/ium) performance trace instructions** - Go to the watchlist - Right click anywhere on the page and choose "Inspect". A panel will open on the right-hand side or bottom of the screen. - In the inspector menu bar, click the three dots icon at right, select settings, make sure “Disable cache (while DevTools is open)” is selected.ma - Along the top line of this panel, you'll see Elements, Console, Sources, etc. Choose Performance (if the panel is on the side, you may need to click the >> button to get there) - In the bar below "Performance", you'll see a record button and a reload button. Click the reload button ("Start profiling and reload page"). - A recording will start and the page will reload. When it's completely finished loading, click the red "record" button or the blue "stop" button to stop the recording. - In the bar where you found the record and reload buttons, click the down arrow button ("save profile") - This opens a dialog that will let you save your recording as a .json file. - Do not upload this file to Phabricator. Please email it to kharlan [at] wikimedia.org and put T197168 in the subject **Firefox instructions** - Go to the watchlist - Right click anywhere on the page and choose "Inspect element". A panel will open on the bottom of the screen. - Along the top line of this panel, you'll see Inspector, Console, Debugger, etc. Choose Performance. - Click the "Start recording performance" in the middle, then reload the page. Once the page finishes loading, click "Stop recording performance". - On the left, you will see a bar with "Recording #1", and a small "save" link next to it. Click the "save" link. - This opens a dialog that will let you save your recording as a .json file. - Do not upload this file to Phabricator. Please email it to kharlan [at] wikimedia.org and put T197168 in the subject

[[ https://www.mediawiki.org/wiki/Topic:Ue4bfpalv9hi25w5 | A user reports very slow load times ]] for Watchlist. The user has a large Watchlist of over 5500 pages, but his load time with filters of 27 seconds seems too long. Trizek, for example, has 4000 pages on his Watchlist, and it loads in just over 3 seconds. The user has provided lots of detail about his filters and configuration. (I've asked him to try the load in safemode and report back.) One reason that I'd like to look into this is that I've noticed that times for the new filters MAY have gotten slower over the last 6 months. See the stats below, which compare times averaged over a week from recently and six months prior (you can click on the dates to check out for yourself). | **When** | **RC Page avg ready** | **Watchlist avg ready** | | [[ https://grafana.wikimedia.org/dashboard/db/rcfilters-performance?orgId=1&from=1526828440288&to=1527357640288&var-interval=10m | May 20-26 ]] | 2.9 secs | 3.88 secs | [[ https://grafana.wikimedia.org/dashboard/db/rcfilters-performance?orgId=1&from=1511712040288&to=1512241240288&var-interval=10m | Nov 26-Dec 2 ]] | 2.5 secs | 3.4 secs | --- ## Summary of issues These are sourced from [Plans to graduate the New Filters on Watchlist out of beta on Talk:Edit Review Improvements/New filters for edit review](https://www.mediawiki.org/w/index.php?title=Topic:Ue4bfpalv9hi25w5&topic_showPostId=ufhj7291s5boc651&fromnotif=1#flow-post-ufhj7291s5boc651) and [Slow and unresponsive on Talk:Edit Review Improvements/New filters for edit review](https://www.mediawiki.org/w/index.php?title=Topic:Tztj44ddxkip6avl&topic_showPostId=ufhd3edibkh1bomd&fromnotif=1#flow-post-ufhd3edibkh1bomd). Thanks also to [Amorymeltzer](https://www.mediawiki.org/wiki/User:Amorymeltzer) who generously took the time to generate a JS profile and shared it with us. The highlights: 1. Some users, but not all, are unable to click on any links on the page while the RC Filters JS loads. This is probably a CSS issue but we need to investigate further. 2. All the bug reports are dealing with larger watchlists and also larger `limit` properties when viewing their watchlists. All the users who submitted feedback are using `limit=500` or `limit=1000`. - Bringing the `limit` down to 100 items seems to bring users back into the range of acceptable performance, but these users are not satisfied with `limit=100`, they need to be able to work with 500-1000 items at a time. 3. The bugs have been reported for the watchlist but [at least one user](https://www.mediawiki.org/w/index.php?title=Topic:Ue4bfpalv9hi25w5&topic_showPostId=ufhi2cdoyp4ptci9#flow-post-ufhi2cdoyp4ptci9) sees similar performance problems on `Special:RecentChanges` 4. The initial request time to get data from the server for the watchlist is slow, it takes at least 4-5 seconds per page request. That means at a minimum each individual filter toggle will take 4-5 seconds. - Testing locally, on a watchlist with 25 items, I have a DOMContentLoaded time of 4.66 seconds, and page load at 30.71s. Toggling filters is between 0.6 and 1.5 seconds. Testing with another account locally with a watchlist of 2500 items, and setting limit to 1000, DOMContentLoaded is 42.77 seconds, and page load is 55.53. Toggling filters on a watchlist of this size takes about 12-13 seconds. - On the watchlist with 2500 items, if I set the limit to 25, the DOMContentLoaded time is 4.73s and the page load is 10.02s, toggling filters takes 2-3 seconds. 5. The initial click on the filters box requires an additional call to load the JS/CSS needed to show the expanded filters interface. In addition to taking a few seconds, this seems to also block user interaction with the page. - We could consider loading this data asynchronously once the initial watchlist has loaded. It’s not a lot of data to add to the page by default. 5. Once the filter UI is open, looking at the JS profiler for toggling the checkboxes on the filters, `setVisibleItems()` (in `mw.rcfilters.dm.FilterGroup.js`) and `setupHighlightContainers()` (in `mw.rcfilters.ui.ChangesListWrapperWidget.js`) pop up as slower functions. We need to do some more investigation of the JS flamegraph to see what other functions are slow and what if anything we can optimize. 6. Our bug report sample size is pretty small. We have ~70k users with this feature enabled but only have a handful of bug reports. It's possible that this issue is more widespread and we haven't seen the feedback, or it's possible that it's limited to users with both larger watchlists and larger `limit` properties set in their query. ## Send us your performance traces If you'd like to help us troubleshoot performance issues on your Watchlist, please copy the text below and paste it into a new comment on this task, and fill in each section. - **URL:** e.g. `https://en.wikipedia.org/wiki/Special:Watchlist?hidemyself=1&hidebots=1&hidecategorization=1&hideWikibase=1&limit=500&days=7&enhanced=1&damaging__likelybad_color=c4&damaging__verylikelybad_color=c5&goodfaith__maybebad_color=c3&urlversion=2&safemode=1` - **Browser and version:** //(e.g. Firefox 60)// - **OS:** //e.g. Ubuntu 18.04// - **CPU:** //e.g. i5 dual-core// - **RAM:** //e.g. 8 GB// - **Number of items in watchlist:** //e.g. 5,000// - **Observations:** //Please include any comments or observations you have.// ### Performance trace: Please ensure that `&safemode=1` is added to your URL before doing the performance trace. **Chrome(/ium) performance trace instructions** - Go to the watchlist - Right click anywhere on the page and choose "Inspect". A panel will open on the right-hand side or bottom of the screen. - In the inspector menu bar, click the three dots icon at right, select settings, make sure “Disable cache (while DevTools is open)” is selected.ma - Along the top line of this panel, you'll see Elements, Console, Sources, etc. Choose Performance (if the panel is on the side, you may need to click the >> button to get there) - In the bar below "Performance", you'll see a record button and a reload button. Click the reload button ("Start profiling and reload page"). - A recording will start and the page will reload. When it's completely finished loading, click the red "record" button or the blue "stop" button to stop the recording. - In the bar where you found the record and reload buttons, click the down arrow button ("save profile") - This opens a dialog that will let you save your recording as a .json file. - Do not upload this file to Phabricator. Please email it to kharlan [at] wikimedia.org and put T197168 in the subject **Firefox instructions** - Go to the watchlist - Right click anywhere on the page and choose "Inspect element". A panel will open on the bottom of the screen. - Along the top line of this panel, you'll see Inspector, Console, Debugger, etc. Choose Performance. - Click the "Start recording performance" in the middle, then reload the page. Once the page finishes loading, click "Stop recording performance". - On the left, you will see a bar with "Recording #1", and a small "save" link next to it. Click the "save" link. - This opens a dialog that will let you save your recording as a .json file. - Do not upload this file to Phabricator. Please email it to kharlan [at] wikimedia.org and put T197168 in the subject

[[ https://www.mediawiki.org/wiki/Topic:Ue4bfpalv9hi25w5 | A user reports very slow load times ]] for Watchlist. The user has a large Watchlist of over 5500 pages, but his load time with filters of 27 seconds seems too long. Trizek, for example, has 4000 pages on his Watchlist, and it loads in just over 3 seconds. The user has provided lots of detail about his filters and configuration. (I've asked him to try the load in safemode and report back.) One reason that I'd like to look into this is that I've noticed that times for the new filters MAY have gotten slower over the last 6 months. See the stats below, which compare times averaged over a week from recently and six months prior (you can click on the dates to check out for yourself). | **When** | **RC Page avg ready** | **Watchlist avg ready** | | [[ https://grafana.wikimedia.org/dashboard/db/rcfilters-performance?orgId=1&from=1526828440288&to=1527357640288&var-interval=10m | May 20-26 ]] | 2.9 secs | 3.88 secs | [[ https://grafana.wikimedia.org/dashboard/db/rcfilters-performance?orgId=1&from=1511712040288&to=1512241240288&var-interval=10m | Nov 26-Dec 2 ]] | 2.5 secs | 3.4 secs | --- ## Summary of issues These are sourced from [Plans to graduate the New Filters on Watchlist out of beta on Talk:Edit Review Improvements/New filters for edit review](https://www.mediawiki.org/w/index.php?title=Topic:Ue4bfpalv9hi25w5&topic_showPostId=ufhj7291s5boc651&fromnotif=1#flow-post-ufhj7291s5boc651) and [Slow and unresponsive on Talk:Edit Review Improvements/New filters for edit review](https://www.mediawiki.org/w/index.php?title=Topic:Tztj44ddxkip6avl&topic_showPostId=ufhd3edibkh1bomd&fromnotif=1#flow-post-ufhd3edibkh1bomd). Thanks also to [Amorymeltzer](https://www.mediawiki.org/wiki/User:Amorymeltzer) who generously took the time to generate a JS profile and shared it with us. The highlights: 1. Some users, but not all, are unable to click on any links on the page while the RC Filters JS loads. This is probably a CSS issue but we need to investigate further. 2. All the bug reports are dealing with larger watchlists and also larger `limit` properties when viewing their watchlists. All the users who submitted feedback are using `limit=500` or `limit=1000`. - Bringing the `limit` down to 100 items seems to bring users back into the range of acceptable performance, but these users are not satisfied with `limit=100`, they need to be able to work with 500-1000 items at a time. 3. The bugs have been reported for the watchlist but [at least one user](https://www.mediawiki.org/w/index.php?title=Topic:Ue4bfpalv9hi25w5&topic_showPostId=ufhi2cdoyp4ptci9#flow-post-ufhi2cdoyp4ptci9) sees similar performance problems on `Special:RecentChanges` 4. The initial request time to get data from the server for the watchlist is slow, it takes at least 4-5 seconds per page request. That means at a minimum each individual filter toggle will take 4-5 seconds. - Testing locally, on a watchlist with 25 items, I have a DOMContentLoaded time of 4.66 seconds, and page load at 30.71s. Toggling filters is between 0.6 and 1.5 seconds. Testing with another account locally with a watchlist of 2500 items, and setting limit to 1000, DOMContentLoaded is 42.77 seconds, and page load is 55.53. Toggling filters on a watchlist of this size takes about 12-13 seconds. - On the watchlist with 2500 items, if I set the limit to 25, the DOMContentLoaded time is 4.73s and the page load is 10.02s, toggling filters takes 2-3 seconds. 5. The initial click on the filters box requires an additional call to load the JS/CSS needed to show the expanded filters interface. In addition to taking a few seconds, this seems to also block user interaction with the page. - We could consider loading this data asynchronously once the initial watchlist has loaded. It’s not a lot of data to add to the page by default. 5. Once the filter UI is open, looking at the JS profiler for toggling the checkboxes on the filters, `setVisibleItems()` (in `mw.rcfilters.dm.FilterGroup.js`) and `setupHighlightContainers()` (in `mw.rcfilters.ui.ChangesListWrapperWidget.js`) pop up as slower functions. We need to do some more investigation of the JS flamegraph to see what other functions are slow and what if anything we can optimize. 6. Our bug report sample size is pretty small. We have ~70k users with this feature enabled but only have a handful of bug reports. It's possible that this issue is more widespread and we haven't seen the feedback, or it's possible that it's limited to users with both larger watchlists and larger `limit` properties set in their query. ## Send us your performance traces If you'd like to help us troubleshoot performance issues on your Watchlist, please copy the text below and paste it into a new comment on this task, and fill in each section. - **URL:** //e.g. `https://en.wikipedia.org/wiki/Special:Watchlist?hidemyself=1&hidebots=1&hidecategorization=1&hideWikibase=1&limit=500&days=7&enhanced=1&damaging__likelybad_color=c4&damaging__verylikelybad_color=c5&goodfaith__maybebad_color=c3&urlversion=2&safemode=1//` - **Browser and version:** //(e.g. Firefox 60)// - **OS:** //e.g. Ubuntu 18.04// - **CPU:** //e.g. i5 dual-core// - **RAM:** //e.g. 8 GB// - **Number of items in watchlist:** //e.g. 5,000// - **Observations:** //Please include any comments or observations you have.// ### Performance trace: Please ensure that `&safemode=1` is added to your URL before doing the performance trace. **Chrome(/ium) performance trace instructions** - Go to the watchlist - Right click anywhere on the page and choose "Inspect". A panel will open on the right-hand side or bottom of the screen. - In the inspector menu bar, click the three dots icon at right, select settings, make sure “Disable cache (while DevTools is open)” is selected.ma - Along the top line of this panel, you'll see Elements, Console, Sources, etc. Choose Performance (if the panel is on the side, you may need to click the >> button to get there) - In the bar below "Performance", you'll see a record button and a reload button. Click the reload button ("Start profiling and reload page"). - A recording will start and the page will reload. When it's completely finished loading, click the red "record" button or the blue "stop" button to stop the recording. - In the bar where you found the record and reload buttons, click the down arrow button ("save profile") - This opens a dialog that will let you save your recording as a .json file. - Do not upload this file to Phabricator. Please email it to kharlan [at] wikimedia.org and put T197168 in the subject **Firefox instructions** - Go to the watchlist - Right click anywhere on the page and choose "Inspect element". A panel will open on the bottom of the screen. - Along the top line of this panel, you'll see Inspector, Console, Debugger, etc. Choose Performance. - Click the "Start recording performance" in the middle, then reload the page. Once the page finishes loading, click "Stop recording performance". - On the left, you will see a bar with "Recording #1", and a small "save" link next to it. Click the "save" link. - This opens a dialog that will let you save your recording as a .json file. - Do not upload this file to Phabricator. Please email it to kharlan [at] wikimedia.org and put T197168 in the subject