Jump to content

Wikipedia:Bots/Requests for approval/KolbertBot

Add links
From Wikipedia, the free encyclopedia
The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA. The result of the discussion was  Approved.

KolbertBot

[edit]

Operator: Jon Kolbert (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 21:01, Thursday, August 10, 2017 (UTC)

Automatic, Supervised, or Manual: Supervised

Programming language(s): Python

Source code available: Pywikibot

Function overview: replace http links with https, if available.

Links to relevant discussions (where appropriate): Why we should convert external links to HTTPS wherever possible

Edit period(s): One-time

Estimated number of pages affected: ~35,000

Namespace(s): mainspace and template

Exclusion compliant (Yes/No): Yes

Function details: Modify the following links

Discussion

[edit]

Should this be approved, what would be the preferred edit summary format? Jon Kolbert (talk) 21:01, 10 August 2017 (UTC)[reply]

The plan to change WebCite links to https://www.webcitation.com/ appears to be incorrect. WebCite is at www.webcitation.org, not www.webcitation.com. —RP88 (talk) 23:55, 10 August 2017 (UTC)[reply]

 Fixed Thank you for spotting that transcription error, it has been amended. Jon Kolbert (talk) 00:27, 11 August 2017 (UTC)[reply]

Sometimes URLs are embedded in archive URLs. For example (non-working):

https://www.webarchive.org.uk/wayback/archive/20100803155857/http://www.theregister.co.uk/

In this case one wouldn't change to https://www.theregister.co.uk/ because it may break the webarchive.org.uk URL which may interpret it as a different URL and unable to find the archive. This isn't a real world example, but there are ones like it. Plus changing to https wouldn't do anything anyway since it's part of the path. -- GreenC 00:52, 11 August 2017 (UTC)[reply]

I do believe that supervising the edits while they take place is the right step to prevent such erroneous replacements - but in any case, should a mistake happen,
https://www.webarchive.org.uk/wayback/archive/20130202200017/http://www.babraham.ac.uk/ and
https://www.webarchive.org.uk/wayback/archive/20130202200017/https://www.babraham.ac.uk/ both do work. I have also tested this with webcitation.org as well.
https://www.webcitation.org/5J1lvgxQH?url=http://fire.prohosting.com/hud607/uncommon/reference/usa/army.html
https://www.webcitation.org/5J1lvgxQH?url=https://fire.prohosting.com/hud607/uncommon/reference/usa/army.html
https://web.archive.org/web/20170727004838/http://www.ubc.ca/
https://web.archive.org/web/20170727004838/https://www.ubc.ca/
In each three cases, both links are fully functional. Jon Kolbert (talk) 01:18, 11 August 2017 (UTC)[reply]
I wonder if this, therefore, confirms all cases? I would be happier to see confirmation from the archive.org website that this is the case. Apart from that, this is a very good bot task - and one that needs doing. I believe that with the link provided we have consensus - and we have done similar tasks in the past (The Guardian springs to mind as one that was recently done, although I can't remember the name of the bot). TheMagikCow (T) (C) 10:40, 11 August 2017 (UTC)[reply]
Fiddling around with URLs in WebCitation links, is kind of sort of pushing the boundaries of WP:COSMETICBOT, likewise with other archive URLs. As the operator of IABot, I would really advise against having your bot handle these and leaving it up to IABot. As a BAG member I see no issues with this task and redundancy never hurts for a supported task.—CYBERPOWER (Chat) 11:01, 11 August 2017 (UTC)[reply]

Jon Kolbert, it should be possible to avoid the archive URLs (of which there are many) by regex'ing URLs from the source including http + the first character preceding, and if that character is "/" skip. -- GreenC 13:57, 11 August 2017 (UTC)[reply]

 Done Perfect, would it also be permissible to use AWB as well? Cheers. Jon Kolbert (talk) 18:47, 11 August 2017 (UTC)[reply]
What would you be use AWB for?—CYBERPOWER (Chat) 14:36, 12 August 2017 (UTC)[reply]
I was originally using Pywikibot with a list generated from Special:LinkSearch, but I find that it'll be easier to manage and sort replacement lists in AWB instead, especially if using Regex. Jon Kolbert (talk) 15:06, 12 August 2017 (UTC)[reply]
Hi Jon, could the regex be run on this testcases page and see what happens? User:GreenC/testcases/kolbert -- GreenC 14:06, 15 August 2017 (UTC)[reply]
The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at WT:BRFA.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:Bots/Requests_for_approval/KolbertBot&oldid=1084321144"