Page MenuHomePhabricator

๐ŸŒ Deal with language fallback and mul in the REST API
Closed, ResolvedPublic13 Estimated Story Points

Description

As a reuser familiar with the concept of Wikibase language fallback, and the special position of "mul" code on Wikidata, I want to get a label or description in the requested language, or a respective language from the fallback chain, so that I can show them in my application.

GET /entities/items/{item_id}/labels_with_language_fallback/{language_code}
GET /entities/items/{item_id}/descriptions_with_language_fallback/{language_code}

GET /entities/properties/{property_id}/labels_with_language_fallback/{language_code}
GET /entities/properties/{property_id}/descriptions_with_language_fallback/{language_code}

All aspects of API behaviour (input validation, error responses, response structure, conditional HTTP header handling, authentication, etc) are to be same as for the respective no-language-fallback APIs unless stated otherwise

  • If there is no label or description defined in the language represented by {language_code} but they're defined in any of the languages defined by the Wikibase language fallback chain for {language_code}, API should respond with a 307 HTTP code, and include a Location header linking to a respective non-fallback resource, e.g. if there is no en label but there is one in mul on item Q123 GET /entities/items/Q123/labels_with_language_fallback/en is intended to result with 307 redirect response with Location: .../entities/items/Q123/labels/mul
  • If there is no label or description in the requested language, and in any of the respective languages defined by Wikibase language fallback chain, API should respond with 404 resource-not-found response.

Task Breakdown Notes

  • copy-paste existing GetLabel use case to GetLabelWithLanguageFallback and create new route handler for the new URL
  • modify use case and route handler to handle language fallback
    • create new interface ItemLabelWithFallbackRetriever
    • use FallbackLabelDescriptionLookupFactory to create a language fallback lookup
    • do not throw Redirect exception, instead return label with fallback language if no label exists in the requested language
    • let RouteHandler handle 200 vs 307 response based on the response's language code (equal to requested language or not?)
  • repeat for the other three routes

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptAug 1 2024, 1:23 PM
Ifrahkhanyaree_WMDE renamed this task from [Placeholder] Deal with MUL in the REST API to Deal with mul in the REST API.Aug 2 2024, 11:12 AM
Ifrahkhanyaree_WMDE updated the task description. (Show Details)

Important from my side: mul should not be treated in a special way. It should just be another language code in the language fallback chain that consumers should fall back to if there is no label available in the language they are requesting. No different than for example de-at falling back to de.
IMHO this means we need an answer to how language fallbacks are handled or not in the REST API. There is T334648 for that.

Main discussion points from meeting:

  • Fallback chain is user agnostic
  • MUL is fallback for ANY missing language label regardless of script
  • We'll stick to how the Action API does it but find a way to do it in a RESTful way
  • Opt 1: item/Q42/label/en --> optional redirect to mul (i..e, like a param)
  • Opt 2: /wikibase/v0/entities/items/{item_id}/labels_with_fallback/{language_code}
  • Opt 3: /wikibase/v0/entities_in_my_language/items/Q42 and list of wanted languages in HTTP header. similar or slightly different response format as it is a different resource
  • Opt 4: /wikibase/v0/entities/$language/items/Q42
  • When asked for item/Q42/labels --> send the normal JSON result without forcing the missing labels into mul or the ones that were mul'ed already with labels

@Lydia_Pintscher we're planning to go with Opt 2 (just wanted to let you know)

Silvan_WMDE renamed this task from Deal with mul in the REST API to Deal with language fallback and mul in the REST API.Sep 19 2024, 10:59 AM

Sorry for late question post polishing but I was re-reading this and digesting it (have to admit I got a little lost in the meeting) the 307 redirect, in that example that's written above, isn't mul the fallback for everything? so, they should technically just get the mul value and not the 307?

Redirect is a way to indicate there is no value in the requested language, but there is something down the fallback chain.
Assuming Q123 has labels: {"mul": 'Rosa Parks"} (that is, no distinct label in en),
If GET ../Q123/labels_with_language_fallback/en returned 200 Rosa Parks client would know the label but wouldn't know if it is label defined in en, or if it came from fallback chain.
It is an expectation that labels coming through fallback chain are distinguishable from the "specifically defined" labels, and that the language code of the label coming from the fallback chain is provided.
Doing GET ../Q123/labels_with_language_fallback/en -> 307 see ../Q123/labels_with_language_fallback/mul fulfills this requirement: following to ../Q123/labels_with_language_fallback/mul will provide the label, and it is clear from the redirect that it is a fallback chain label, and that the language in which that "fallback label" was defined is "mul"

Ah gotcha. So, the situation would be -

I am a user who wants labels with a language fallback specifically in EN.

vs

I am a user who wants labels with a language fallback no matter what i.e., in mul

First case they'd get the 307 if there was no en label either.

I think yes but it is kind of the other way round.
Client aware of language fallback always wants a label in a particular language. They accept though they might get label defined not for the requested language but for a fallback language, e.g. mul.

and what would the first case get? just a 404?

apologies for the million questions

questions are good, I think we have not created a clear vocabulary around this yet, that's all.
I will not directly respond to your question, as I don't think first case and second one in your original question differ. But hopefully this helps

Let's take item Q123, which has no EN label but MUL label "Rosa Parks"

So if I wanted EN label and it must be the label defined for EN, I would be going to non-fallback-aware API IMO: ../Q123/labels/en. In the example it would give me 404.
If I ask ../Q123/labels_with_language_fallback/en I say "Give me a label for Q123 preferably in EN but I will take whatever you think is a good enough label if there is no EN one", and in example case API would give me the MUL one

In case of another example item Q321 which only had DE label "Kartoffelpuffer" (i.e. no MUL label), if I asked ../Q321/labels_with_language_fallback/en I'd get 404, as there is no EN label, neither any other in the language fallback chain (specifically, no MUL label)

Yeah this makes sense, but in which of the cases does the 307 come in? This one?

If I ask ../Q123/labels_with_language_fallback/en I say "Give me a label for Q123 preferably in EN but I will take whatever you think is a good enough label if there is no EN one", and in example case API would give me the MUL one

yes, this one.
307 in this case is API's way to communicate: I don't have a label in language you'd have preferred but I do have a label in this language (e.g. MUL) that per Wkibase language fallback chain might still work for you

Silvan_WMDE set the point value for this task to 13.
Dima_Koushha_WMDE renamed this task from Deal with language fallback and mul in the REST API to ๐ŸŒ Deal with language fallback and mul in the REST API.Sep 27 2024, 10:02 AM

After discussing with Lydia - We don't need to do the endpoints for aliases

Tested with

  1. GET items/id/labels_with_language_fallback en-us here: https://wikidata.beta.wmflabs.org/wiki/Q626665 and correctly gave me a 307 with the location header pointing to /labels/en
  1. GET properties/id/descriptions_with_language_fallback/de and got a 307 and location pointing to /descriptions/en
  1. GET items/id/descriptions_with_language_fallback/en and got a 404 as expected
  1. GET/ items/id/labels_with_language_fallback de and gave me the de label that existed

Change #1079304 had a related patch set uploaded (by Jakob; author: Jakob):

[mediawiki/extensions/Wikibase@master] REST: Add missing tests for language fallback routes

https://gerrit.wikimedia.org/r/1079304

Change #1079304 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] REST: Add missing tests for language fallback routes

https://gerrit.wikimedia.org/r/1079304