Page MenuHomePhabricator

Update web clients to source Wikidata descriptions from local wiki where available
Closed, ResolvedPublic5 Estimated Story Points

Description

In T184000 a mechanism was provided for wikidata descriptions to be updated by editors via a magic word. A prop 'description' and prop 'descriptionsource' was added to pages. Currently the mobile web client does not make use of these.

Acceptance criteria

  • Ensure MobileFrontend/Minerva/RelatedArticles source from description rather than Wikidata if available in all PHP and API calls.

Developer notes

https://en.wikipedia.org/w/api.php?action=query&prop=description&titles=Wagon_Train

returns

{
    "batchcomplete": "",
    "query": {
        "normalized": [
            {
                "from": "Wagon_Train",
                "to": "Wagon Train"
            }
        ],
        "pages": {
            "1071760": {
                "pageid": 1071760,
                "ns": 0,
                "title": "Wagon Train",
                "description": "Western television series aired 1957-1965",
                "descriptionsource": "local"
            }
        }
    }
}

In the server side we'll need to get page props

$out->getProperty('description')

Testing

You'll need to use the beta cluster for this.

https://en.m.wikipedia.beta.wmflabs.org/wiki/Albert_Einstein has a wikidata description that should match https://wikidata.beta.wmflabs.org/wiki/Special:EntityPage/Q356553

https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF&venotify=created&mobileaction=toggle_view_mobile
is associated with the Wikidata article https://wikidata.beta.wmflabs.org/wiki/Q450237.

Testing overriden descriptions

Edit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF and ensure the following text is present:

{{SHORTDESC:Woof woof}}

Verify

  • When searching for "Woof woof" the description shows as Woof woof.
  • When viewing the page "Woof woof" the description under the heading is "woof woof"
  • When you visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Monkey and look at related articles, the card "Woof woof" has the description "woof woof"

Testing the blanking behaviour

Edit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF and ensure the following text is present:

{{SHORTDESC:}}

Verify

  • When searching for "Woof woof" the description doesn't show from wikidata shows
  • When viewing the page "Woof woof" the description under the heading is blank~ from wikidata shows
  • When you visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Monkey and look at related articles, the card "Woof woof" has no description.~ from wikidata shows

[JR: Misunderstood how this works]

Testing the wikidata behaviour

Edit https://en.m.wikipedia.beta.wmflabs.org/w/index.php?title=WOOF_WOOF and ensure SHORTDESC is NOT present in the wikitext.

Verify

  • When searching for "Woof woof" the description shows with the value on wikidata (currently Meow)
  • When viewing the page "Woof woof" the description under the heading shows with the value on wikidata (currently Meow)
  • When you visit https://en.m.wikipedia.beta.wmflabs.org/wiki/Monkey and look at related articles, the card "Woof woof" has the description "Meow"

Screen Shot 2018-04-10 at 8.52.35 AM.png (274×419 px, 16 KB)

Wikis without local override

  • We'll want to verify this is also working for wikis which do not have a local override.

I've linked up Spain articles on the beta cluster to a Wikidata entry

Event Timeline

In the server side we'll need to get page props

$out->getProperty('description')

The property name is wikibase-shortdesc. Querying that will only work when there is a local override, though. I suppose Wikibase should provide some sort of helper method... how are you handling it on the server side currently?

Is this task for web only or should we fan out from here to other clients as well?

Jdlrobson renamed this task from Update client to source Wikidata descriptions from local wiki where available to Update web clients to source Wikidata descriptions from local wiki where available.Mar 28 2018, 11:15 PM
ovasileva triaged this task as Medium priority.Mar 29 2018, 3:08 PM
ovasileva moved this task from Triaged but Future to Upcoming on the Web-Team-Backlog board.
ovasileva raised the priority of this task from Medium to High.Apr 3 2018, 11:43 AM

Discussed in estimation. Note we should request both description (new) and the pageterms (already queried) in server and client usages in case the project that is running the code doesn't have description props.

Jdlrobson set the point value for this task to 5.Apr 3 2018, 4:22 PM

We're hitting 3 repos and setting up wikidata descriptions locally makes testing difficult so we've estimated by 5.

I suppose Wikibase should provide some sort of helper method...

Done in https://gerrit.wikimedia.org/r/c/423244/ (well, done once it gets merged).
See https://gerrit.wikimedia.org/r/c/423245/ for an example of using it.

Note we should request both description (new) and the pageterms (already queried) in server and client usages in case the project that is running the code doesn't have description props.

MobileFrontend is documented as using the release branch compatibility model, in which case that shouldn't be possible.

@Tgr I'm not sure I'm following are you saying we can ignore pageterms now? It was my understanding that pageterms will refer to the value inside Wikidata and pageprops the local version. Is that not correct? Can we ignore pageterms altogether now?

If you call the description API, it will look for a local override, and fall back to the Wikidata description if not found. This is true with or without the pending patch.

The patch makes the functionality of the API available from PHP, so you can do something like

$wikibaseClient = WikibaseClient::getDefaultInstance();
$descriptionLookup = $wikibaseClient->getStore()->getDescriptionLookup();
$description = $descriptionLookup->getDescription( $title, [ DescriptionLookup::SOURCE_LOCAL, DescriptionLookup::SOURCE_CENTRAL ] );

(where the last parameter tells the description service to use the local description and fall back to the Wikidata one).

(FWIW the description API and the pageterms API have different behavior for language fallback: if you call them on the German Wikipedia, and the page has a Wikidata description in English, but not in German, the pageterms API will return the English description and the description API won't return anything, which is IMO the more correct behavior for use cases like showing the description as a navigation aid.)

Change 425261 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/extensions/MobileFrontend@master] Replace "pageterm" wikidata API calls with "description" API

https://gerrit.wikimedia.org/r/425261

Change 425261 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Replace "pageterm" wikidata API calls with "description" API

https://gerrit.wikimedia.org/r/425261

Added testing instructions. Looks like we're 1/3 of the way to doing this! w00t!

Change 425768 had a related patch set uploaded (by Jdrewniak; owner: Jdrewniak):
[mediawiki/extensions/RelatedArticles@master] Replace pageterms with description API

https://gerrit.wikimedia.org/r/425768

The patches above replace client-side calls to pageterms API with the description API in mobilefrontend and RelatedArticles.

I wasn't able to find any occurrences of the pageterms API in Minerva, also, I wasn't able to find any server-side usages of the pageterms API in these codebases.

Am I missing something?

https://gerrit.wikimedia.org/r/425261 would have taken care of Minerva and MobileFrontend client side code.
The server side usage is inside includes/MobileFrontend.body.php :

ExtMobileFrontend::getWikibaseDescription

We'll probably want to update the function signature and the caller (since the new description is not coming from the Wikidata item) so that it also looks in the ParserOutput.

public static function onOutputPageParserOutput( $outputPage, ParserOutput $po ) {

I can take a look at that today if that helps.. I'm pretty familiar with this code and it's hard to test.

Change 425911 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] Short description can now come from wikitext

https://gerrit.wikimedia.org/r/425911

Change 425768 merged by jenkins-bot:
[mediawiki/extensions/RelatedArticles@master] Replace pageterms with description API

https://gerrit.wikimedia.org/r/425768

Piotr to provide guidance on the PHPy bits.

Just out of curiosity: Note that T184000 only applies to enwiki. What are the implications of this for the implementation and testing of the present task in all the other languages (and non-Wikipedia wikis)?

Change 425911 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] Short description can now come from wikitext

https://gerrit.wikimedia.org/r/425911

Jdlrobson updated the task description. (Show Details)
Jdlrobson added subscribers: ABorbaWMF, pmiazga.

Over to you @ABorbaWMF !
Note testing should happen on the beta cluster, but as well as https://en.m.wikipedia.beta.wmflabs.org/wiki/Spain you will also need to test on the wikivoyage version of the beta cluster (https://en.m.wikivoyage.beta.wmflabs.org/wiki/Spain) - testing steps should be concise but let me know if anything is not clear.

Things seem to be working with the exception of the blanking portion. I am still seeing the description on screen after blanking. @Jdlrobson, I'm not sure if you want to kick the whole ticket back or just break out the blanking portion in a separate bug.

@Tgr is blanking supporting?

When I set {{SHORTDESC:}} in the wikitext it doesn't seem to clear the description. Is that expected?
https://en.m.wikipedia.beta.wmflabs.org/w/api.php?action=query&format=json&prop=description&generator=prefixsearch&ppprop=displaytitle&piprop=thumbnail&pithumbsize=80&pilimit=15&wbptterms=description&redirects=&gpssearch=Spain&gpsnamespace=0&gpslimit=15

EDIT: Per T184000 this is the correct behaviour "If the magic word isn't used on the page, or if the description is blank, then it should show the description from Wikidata."

Jdlrobson updated the task description. (Show Details)

Do you want to take a look at this and sign this off given Olga is away?

Oh cool. I tried the above scenarios on a few devices and browsers. I am seeing the descriptions in the right places.

image.png (984×540 px, 110 KB)

image.png (1×720 px, 160 KB)

Went through the testing steps as well. Looks good to me.