Page MenuHomePhabricator
Create Task
Maniphest T190254

Remove irrelevant sections from source article for translation
Closed, ResolvedPublic
Actions

Assigned To
santhosh
Authored By
santhosh
Mar 21 2018, 9:33 AM
Tags
Referenced Files
F16927262: Screen Shot 2018-04-11 at 9.55.52 PM.png
Apr 12 2018, 5:04 AM
F16927253: Screen Shot 2018-04-11 at 9.56.08 PM.png
Apr 12 2018, 5:04 AM
F16927206: Screen Shot 2018-04-11 at 9.46.01 PM.png
Apr 12 2018, 5:04 AM
F16927212: Screen Shot 2018-04-11 at 9.46.42 PM.png
Apr 12 2018, 5:04 AM
F15942234: image.png
Mar 21 2018, 9:33 AM
F15942248: image.png
Mar 21 2018, 9:33 AM
Subscribers
Aklapper
Etonkovidova
gerritbot
Nikerabbit
Pginer-WMF
santhosh

Description

Pages with project specific content need not be presented as section to translate. For example a section like below in the top of an article is not relevant in the context of translating that article

image.png (122×1 px, 24 KB)

Another example:

image.png (448×315 px, 34 KB)

CX1 had a configuration file listing the selectors for this kind of removable sections. See https://github.com/wikimedia/mediawiki-extensions-ContentTranslation/blob/master/modules/source/conf/common.json#L2

In CX2, I propose to do this removal of irrelevant sections at cxserver while parsing the page content. This makes the client code less crowded. Also remove the need of serving the configuration file to every client.

Details

SubjectRepoBranchLines +/-
Add support for removing sections by transclusion namemediawiki/services/cxservermaster+64 -4
Remove Indicator elements from the parsed source articlemediawiki/services/cxservermaster+3 -1
CX2: Do not fetch configuration jsonmediawiki/extensions/ContentTranslationmaster+4 -37
Remove irrelavant sections from parsed pagemediawiki/services/cxservermaster+116 -7
Customize query in gerrit

Related Objects

Event Timeline

santhosh created this task.Mar 21 2018, 9:33 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 21 2018, 9:33 AM
santhosh triaged this task as Medium priority.Mar 21 2018, 9:34 AM
santhosh moved this task from Needs Triage to CX2 on the ContentTranslation board.
Pginer-WMF added a project: Language-2018-Jan-Mar.Mar 21 2018, 9:53 AM
santhosh mentioned this in T190203: CX2: Hidden categories are adapted and displayed.Mar 21 2018, 10:20 AM
Arrbee assigned this task to santhosh.Mar 27 2018, 7:08 AM
Arrbee moved this task from Backlog to In Progress on the Language-2018-Jan-Mar board.
santhosh moved this task from In Progress to In Review on the Language-2018-Jan-Mar board.Mar 28 2018, 7:01 AM
santhosh added a project: Patch-For-Review.Mar 28 2018, 8:02 AM

Patch: https://gerrit.wikimedia.org/r/c/421868/

Pginer-WMF added a project: Language-2018-Apr-June.Apr 3 2018, 8:18 AM
Pginer-WMF moved this task from Backlog to In Review on the Language-2018-Apr-June board.
Pginer-WMF removed a project: Language-2018-Jan-Mar.Apr 3 2018, 8:26 AM
gerritbot subscribed.Apr 4 2018, 7:00 AM

Change 421868 merged by jenkins-bot:
[mediawiki/services/cxserver@master] Remove irrelavant sections from parsed page

https://gerrit.wikimedia.org/r/421868

Nikerabbit moved this task from In Review to QA on the Language-2018-Apr-June board.Apr 4 2018, 7:01 AM
Nikerabbit removed a project: Patch-For-Review.
Nikerabbit subscribed.Apr 4 2018, 7:17 AM

One easy method to find articles with timelines is to search insource:/\<timeline\>/. I tested on en:World Heritage Site.

gerritbot added a comment.Apr 6 2018, 11:57 AM

Change 424574 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/extensions/ContentTranslation@master] CX2: Do not fetch configuration json

https://gerrit.wikimedia.org/r/424574

gerritbot added a project: Patch-For-Review.Apr 6 2018, 11:57 AM
santhosh moved this task from QA to In Review on the Language-2018-Apr-June board.Apr 6 2018, 11:59 AM
gerritbot added a comment.Apr 9 2018, 1:12 PM

Change 424574 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] CX2: Do not fetch configuration json

https://gerrit.wikimedia.org/r/424574

ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-04-10 (1.31.0-wmf.29)).Apr 9 2018, 2:01 PM
Arrbee moved this task from In Review to QA on the Language-2018-Apr-June board.Apr 10 2018, 7:04 AM
• Petar.petkovic removed a project: Patch-For-Review.Apr 10 2018, 10:59 PM
Etonkovidova subscribed.Apr 12 2018, 5:04 AM

I compared cx-testing and cx2-testing.
In cx-testing the project specific content is not present at all:

Screen Shot 2018-04-11 at 9.46.01 PM.png (477×915 px, 187 KB)

In cx2-testing - the abbreviated templates are displayed and can be "translated":

Screen Shot 2018-04-11 at 9.46.42 PM.png (513×926 px, 134 KB)

@santhosh - the current behavior in cx2-testing is correct? Or such templates are a subject to "removal of irrelevant sections at cxserver while parsing the page content"?

As somewhat extreme example - the article 'Routes in Train Simulator 2012' is currently nominated for deletion. Shouldn't a translator be warned on the translation page about it?
cx2-testing (displays the template)

Screen Shot 2018-04-11 at 9.56.08 PM.png (606×947 px, 77 KB)

cx-testing does not provide any information that the article is nominated for deletion:

Screen Shot 2018-04-11 at 9.55.52 PM.png (331×908 px, 53 KB)

santhosh added a subscriber: Pginer-WMF.Apr 19 2018, 3:54 AM

In general template handling is a big work we have not addressed in CX2.

In this ticket, all the content that is not part of the article, but about the article(meta content ) is removed from page. Since they don't need to be translated.

In cx2-testing - the abbreviated templates are displayed and can be "translated":

In quick look, it seems these things should be removed. Need to check them. Since removing a particular meta content is just based on a json configuration now, I guess it would be nice to address them in new tickets, and add the entries to the json.

As somewhat extreme example - the article 'Routes in Train Simulator 2012' is currently nominated for deletion. Shouldn't a translator be warned on the translation page about it?

This is an important thing we can consider. But it would be a new feature in CX2. cc @Pginer-WMF

Pginer-WMF added a comment.Apr 19 2018, 9:30 AM
In T190254#4141719, @santhosh wrote:

As somewhat extreme example - the article 'Routes in Train Simulator 2012' is currently nominated for deletion. Shouldn't a translator be warned on the translation page about it?

This is an important thing we can consider. But it would be a new feature in CX2. cc @Pginer-WMF

Makes sense. This can help users to avoid wasting time and propagating problematic content. I created a separate ticket for this (T192530).

gerritbot added a comment.Apr 19 2018, 10:24 AM

Change 427630 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/services/cxserver@master] Remove Indicator elements from the parsed source article

https://gerrit.wikimedia.org/r/427630

gerritbot added a project: Patch-For-Review.Apr 19 2018, 10:24 AM
santhosh mentioned this in T192541: Remove Wikipedia progress and quality related templates from source article.Apr 19 2018, 11:14 AM
gerritbot added a comment.Apr 19 2018, 11:14 AM

Change 427642 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/services/cxserver@master] Add support for removing sections by transclusion name

https://gerrit.wikimedia.org/r/427642

• Petar.petkovic moved this task from QA to In Review on the Language-2018-Apr-June board.Apr 19 2018, 11:31 AM
gerritbot added a comment.Apr 25 2018, 2:09 PM

Change 427630 merged by jenkins-bot:
[mediawiki/services/cxserver@master] Remove Indicator elements from the parsed source article

https://gerrit.wikimedia.org/r/427630

gerritbot added a comment.Apr 30 2018, 8:48 AM

Change 427642 merged by jenkins-bot:
[mediawiki/services/cxserver@master] Add support for removing sections by transclusion name

https://gerrit.wikimedia.org/r/427642

Arrbee moved this task from In Review to QA on the Language-2018-Apr-June board.May 2 2018, 7:03 AM
• Petar.petkovic removed a project: Patch-For-Review.May 2 2018, 8:14 AM
Etonkovidova closed this task as Resolved.May 9 2018, 11:57 PM

Checked in cx2-testing - article's sections such as "Additional citation" (e.g. {{Refimprove|date=November 2016}} in "Astrology and the classical elements") and others are not present in the source article on CX page. A specific cases - an article was nominated for deletion - will be addressed in T192530: CX2: Warn the user when translating an article that is nominated for deletion.

• Petar.petkovic moved this task from QA to Done on the Language-2018-Apr-June board.May 14 2018, 9:11 PM
santhosh mentioned this in T197859: Improve non-translatable content skiplist mechanism in cxserver.Jun 21 2018, 11:57 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits