User:Amorymeltzer/s-index

The number of enWiki sysops has been in a steady decline for years. It is a recurring topic at WT:RFA, a common thread in RfA reform proposals, and has been extensively covered at the Signpost and elsewhere; many causes and solutions have been offered and discussed. The s-index is a metric for assessing the activity of the entire corps of sysops. It is analogous to the h-index for academics, which "attempts to measure both the productivity and citation impact of the publications" of an academic, measuring the number of papers h with at least h citations. An h-index of five means the author has five papers that have been cited at least five times each, while an h-index of 11 means the author has published 11 with at least 11 citations. The intent of the h-index is to measure both output and impact, not just quantity of either. In essence, the s-index treats enWiki as the scholar, with each sysop as a paper and logged actions as each sysop's "citation" count. A similar metric used in cycling is the Eddington number.

The code for this project pulls each month's data from XTools' AdminStats and processes those values. It then calculates the s-index for the specified option, and graphs it using R. The idea is to observe how the project's s-index has changed and is changing over time, hopefully to provide some context. Below are a number of these graphs, each looking at a different range of data. The monthly graphs show the s-index for each month, while the annual graphs show the s-index for each full calendar year (i.e. through 2017). The rolling graphs show a rolling lookback for a specific number of months at a time, e.g. "rolling 3mos" shows the s-index for the each three-month period available. Each graph shows the s-index both with and without bots, and a LOESS regression is also shown over the data. An identical graph also exists for each that shows the total number of admin actions in that period.

Graphs

Monthly

Rolling

3 months

6 months

12 months

Annual

• Note: Data through 2017

Takeaways

• The monthly s-index charts make sense in the broader picture of sysop activity, in particular the peak in the late 2000s.
• The project hit a low point, both in terms of s-index and actions, around 2014 or 2015.
  • The s-index has not changed significantly since then, and is now potentially increasing, albeit slowly.
• Since 2008 and especially recently, bots have become an increasingly important source of admin actions. Nonbot actions have slowly increased and are at roughly 2006 levels while bot sysop actions have increased rapidly, likely leading total actions (including bots) to a record level in the near future.
  • Most recent sysopbot actions are from just two bots, thus leaving the s-index comparatively unchanged.

Caveats and drawbacks

• Data begin January 2005, as that is the first full month with logs.
• The h-index is itself flawed, so while potentially interesting this s-index really doesn't actually mean anything and it certainly isn't clear what a "good" number would be.
• This relies on data from the AdminStats tool from XTools, which is most excellent; they are better at what they do than I am at what I do, so if there's an issue there's a good chance I'm wrong about something.
  • AdminStats includes (re)(un)blocks, (un)deletions (including revdels), (re)(un)protections, user rights changes, and imports. Anything else (edit filter changes, renames, merges, massmessages) isn't counted as a sysop action.
  • AdminStats includes only the above logged items, so other typical sysop activities, such as closing deletion discussions, are not counted. This is a common issue with sysop-related metrics.
  • AdminStats handles renames automatically, but I have to redownload the data if someone changes their name. That's not too difficult, and I have a few built-in checks to ensure the data are otherwise unchanged, but this may be ever-so-slightly out of date until I am aware of a sysop rename.
• Not all sysop actions are equal. Revision deletion is per page, so redacting one edit or 42 edits may be just one logged action, while redacting four edits, each on a different page, will be four actions.
• Users who are not sysops may appear to have taken actions in the past, such as by moving a protected page or overwriting a redirect.
• WMF staff member accounts are not sysops but nonetheless occasionally take sysop actions. Occasionally, these may affect the s-index, but only rarely and only ever by a single point, so they are left in.