In fact, it would appear that different bots and users have managed to upload every single Windows 7 stock image, even though all of them should be prevented by this filter.

The oldest was uploaded on 5 November (https://commons.wikimedia.org/wiki/File:등대.jpg), the latest today (https://commons.wikimedia.org/wiki/File:Gggb.jpg).

OK, I see… I can reproduce the issue locally with $wgAbuseFilterParserClass = 'AbuseFilterCachingParser';. This was enabled for Commons with https://gerrit.wikimedia.org/r/#/c/316354/ on 2016-10-17. I suppose there's something about this specific filter that makes the new parser produce wrong results.

That config change has already caused different filters to break twice, and was reverted and re-enabled: T148660, T148673. I think we should revert it again ASAP, and keep it reverted this time until @ori or @vvv prove that it's not completely broken (e.g. by writing a test suite for it).

Patch deployed by @thcipriani and submitted to Gerrit as https://gerrit.wikimedia.org/r/327269.

Can you file a separate task about the bug in AbuseFilterCachingParser?

All production wikis were affected by this issue, in the following periods. But other than Commons' filter 31, we can't say which filters have been affected, until someone actually debugs the problem. For now we just reverted to a known good version of the code.

We should announce to the communities that this happened. (Perhaps after we know which filters were broken, if anyone is planning to investigate.)

Ideally, we would also re-run the filters for all the actions in this time period, but there is currently no mechanism to do this and it would be a significant effort to develop and test one (in the ballpark of a person-week, I guess), and it could take a while to run such a script. (While we can't retroactively prevent actions from being made, we could still tag the edits and add entries to abuse log, etc.) @greg @Jdforrester-WMF Could you consider this and decide whether it's worth it?

In T153217#2873920, @Legoktm wrote:

Can you file a separate task about the bug in AbuseFilterCachingParser?

Thanks for the reminder, filed T153251 for AbuseFilter, let's keep this task about the Wikimedia perspective.

Filters can be hand-tested on recent changes, maybe just ask admins to do that? It wouldn't leave a tag/log audit trail though.

I don't think it's worth trying to retrospectively re-run AbuseFilters (especially as it looks like many of the important ones deny upload/creation which would be impossible to "tag"). However, I think @Tgr is right and we should highlight this to sysops (maybe just in Tech/News, maybe in a more prominent way?).

Something like this for next issue of Tech News?

Some abuse filters for uploaded files have not worked as they should. We don't know exactly which filters didn't work yet. This means some files that filters should have prevented from being uploaded were uploaded to the wikis. MediaWiki.org and Testwiki have been affected since October 13. Commons and Meta have been affected since October 17. Other wikis have been affected since November 17.

I think that's fine, although it's possible that other filters (not just those about action='upload') were also affected – we literally have no idea.

@vvv did the debugging and found out the reason: T153251#2884699. In short, the problem affected filters using variadic functions (functions that can take any number of arguments, like contains_any) where there is a trailing comma , before the closing parenthesis ).

The following wikis and the following filters were likely affected. I ran this SQL query on all databases to find them:

select 'foowiki' as wiki, af_id
from foowiki.abuse_filter
where af_pattern rlike ',[ \t\v\f\r\n]*(/\\*.*\\*/[ \t\v\f\r\n]*)*\\)'
and af_enabled and not af_deleted

I think we should send an extra MassMessage to village pumps on these 32 wikis, with a list of their affected filters. @Johan Can you help?

FYI, enwiki filters 755 and 756 are mine and they're not affected AFAICT: they don't use variadic functions and the ,) arises inside a regex. May I also suggest sending any enwiki message also to the Edit Filter Noticeboard? Thanks.

In T153217#2888136, @matmarex wrote:

The following wikis and the following filters were likely affected. I ran this SQL query on all databases to find them:

select 'foowiki' as wiki, af_id
from foowiki.abuse_filter
where af_pattern rlike ',[ \t\v\f\r\n]*(/\\*.*\\*/[ \t\v\f\r\n]*)*\\)'
and af_enabled and not af_deleted

Wiki name	Filter number
arwiki	64
commonswiki	31
cswiki	2
dawiki	11
elwiki	3
enwiki	755
enwiki	756
eswiktionary	6
fawiki	3
fawiki	9
fawiki	19
fawiki	33
fawiki	120
fawikibooks	10
fawikinews	11
fawikinews	14
fawikinews	23
fawikiquote	2
fawikivoyage	2
fawikivoyage	8
fawiktionary	9
fiwiki	124
gagwiki	1
hiwiki	43
hiwiki	61
hiwiki	80
jawiki	18
jawikibooks	2
jawikiquote	1
jawikisource	1
jawikisource	2
kkwiki	3
metawiki	114
mlwiki	3
mrwiki	21
nnwiki	5
nowiki	4
ruwiki	14
svwiki	85
testwiki	81
testwiki	108
zh_yuewiki	3
zh_yuewiki	20
zhwiki	6
zhwiki	31
zhwiki	37
zhwiki	38
zhwiki	39
zhwiki	53
zhwiki	55
zhwiki	92
zhwiki	191
zhwikinews	12

I don't think commonswiki filter 31 is not affected, it is still working and disallowing frequently uploaded non-free files.

@Pokefan95 It is working now, after we reverted the buggy changes, but it used to be broken, and copies of these files were uploaded (see a list in my earlier comment).

@matmarex Sorry, was on holiday break and didn't see this ping until now. Still want my help?

Eh, so was I, and I kind of forgot about this to be honest. Perhaps we don't need to do that, this task was linked from that Tech News and anyone interested would have found it.