Skip to content

Display names data experimentation #6880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Display names data experimentation #6880

wants to merge 7 commits into from
+326 −0

Conversation

sffc
Copy link
Member

@sffc sffc commented Aug 19, 2025

Number of unique entries per locale with language fallback:

af: 294
af-NA: 0
agq: 222
ak: 294
am: 294
ar: 294
ar-AE: 0
ar-BH: 0
ar-DJ: 0
ar-DZ: 0
ar-EG: 0
ar-EH: 0
ar-ER: 0
ar-IL: 0
ar-IQ: 0
ar-JO: 0
ar-KM: 0
ar-KW: 0
ar-LB: 0
ar-LY: 3
ar-MA: 0
ar-MR: 0
ar-OM: 0
ar-PS: 0
ar-QA: 0
ar-SA: 5
ar-SD: 0
ar-SO: 0
ar-SS: 0
ar-SY: 0
ar-TD: 0
ar-TN: 0
ar-YE: 0
as: 294
asa: 220
ast: 289
az: 294
az-IQ: 0
az-Arab: 1
az-Arab-TR: 0
az-Cyrl: 284
bal-Latn: 295
bas: 219
be: 294
be-tarask: 0
bem: 1
bez: 222
bg: 294
bgc: 2
bgn: 110
bgn-AE: 0
bgn-AF: 0
bgn-IR: 0
bgn-OM: 0
bho: 2
blo: 295
bm: 222
bn: 294
bn-IN: 4
bo: 12
bo-IN: 1
br: 293
brx: 294
bs: 294
bs-Cyrl: 294
bss: 1
ca: 294
ca-AD: 0
ca-ES-valencia: 0
ca-FR: 0
ca-IT: 0
ccp: 288
ccp-IN: 0
ce: 289
ceb: 294
cgg: 221
cho: 1
chr: 294
cic: 1
ckb: 291
ckb-IR: 0
cs: 294
csw: 10
cv: 293
cy: 294
da: 294
da-GL: 0
dav: 222
de: 294
de-AT: 1
de-BE: 0
de-CH: 7
de-IT: 0
de-LI: 0
de-LU: 0
dje: 221
doi: 11
dsb: 294
dua: 1
dyo: 108
dz: 286
ebu: 222
ee: 281
ee-TG: 0
el: 294
el-polyton: 97
el-CY: 0
en: 295
en-001: 9
en-150: 0
en-AE: 0
en-AG: 0
en-AI: 0
en-AS: 0
en-AT: 0
en-AU: 6
en-BB: 0
en-BE: 0
en-BI: 0
en-BM: 0
en-BS: 0
en-BW: 0
en-BZ: 0
en-CA: 21
en-CC: 0
en-CH: 0
en-CK: 0
en-CM: 0
en-CX: 0
en-CY: 0
en-CZ: 0
en-DE: 0
en-DG: 0
en-DK: 0
en-DM: 0
en-ER: 0
en-ES: 0
en-FI: 0
en-FJ: 0
en-FK: 0
en-FM: 0
en-FR: 0
en-GB: 0
en-GD: 0
en-GG: 0
en-GH: 0
en-GI: 0
en-GM: 0
en-GS: 0
en-GU: 0
en-GY: 0
en-HK: 0
en-HU: 0
en-ID: 0
en-IE: 0
en-IL: 0
en-IM: 0
en-IN: 0
en-IO: 0
en-IT: 0
en-JE: 0
en-JM: 0
en-KE: 0
en-KI: 0
en-KN: 0
en-KY: 0
en-LC: 0
en-LR: 0
en-LS: 0
en-MG: 0
en-MH: 0
en-MO: 0
en-MP: 0
en-MS: 0
en-MT: 0
en-MU: 0
en-MV: 0
en-MW: 0
en-MY: 0
en-NA: 0
en-NF: 0
en-NG: 0
en-NL: 0
en-NO: 0
en-NR: 0
en-NU: 0
en-NZ: 0
en-PG: 0
en-PH: 0
en-PK: 0
en-PL: 0
en-PN: 0
en-PR: 0
en-PT: 0
en-PW: 0
en-RO: 0
en-RW: 0
en-SB: 0
en-SC: 0
en-SD: 0
en-SE: 0
en-SG: 0
en-SH: 0
en-SI: 0
en-SK: 0
en-SL: 0
en-SS: 0
en-SX: 0
en-SZ: 0
en-TC: 0
en-TK: 0
en-TO: 0
en-TT: 0
en-TV: 0
en-TZ: 0
en-UG: 0
en-UM: 0
en-VC: 0
en-VG: 0
en-VI: 0
en-VU: 0
en-WS: 0
en-ZA: 0
en-ZM: 0
en-ZW: 0
en-Dsrt: 273
eo: 222
es: 294
es-419: 26
es-AR: 3
es-BO: 3
es-BR: 0
es-BZ: 0
es-CL: 4
es-CO: 3
es-CR: 3
es-CU: 0
es-DO: 3
es-EA: 0
es-EC: 3
es-GQ: 0
es-GT: 3
es-HN: 3
es-IC: 0
es-MX: 17
es-NI: 3
es-PA: 3
es-PE: 3
es-PH: 0
es-PR: 1
es-PY: 3
es-SV: 1
es-US: 18
es-UY: 0
es-VE: 3
et: 295
eu: 294
ewo: 222
fa: 294
fa-AF: 89
ff: 222
ff-BF: 0
ff-CM: 0
ff-GH: 0
ff-GM: 0
ff-GN: 0
ff-GW: 0
ff-LR: 0
ff-MR: 0
ff-NE: 0
ff-NG: 0
ff-SL: 0
ff-Adlm: 293
ff-Adlm-BF: 0
ff-Adlm-CM: 0
ff-Adlm-GH: 0
ff-Adlm-GM: 0
ff-Adlm-GW: 0
ff-Adlm-LR: 0
ff-Adlm-MR: 0
ff-Adlm-NE: 0
ff-Adlm-NG: 0
ff-Adlm-SL: 0
ff-Adlm-SN: 0
fi: 295
fil: 294
fo: 295
fo-DK: 0
fr: 294
fr-BE: 1
fr-BF: 0
fr-BI: 0
fr-BJ: 0
fr-BL: 0
fr-CA: 40
fr-CD: 0
fr-CF: 0
fr-CG: 0
fr-CH: 0
fr-CI: 0
fr-CM: 0
fr-DJ: 0
fr-DZ: 0
fr-GA: 0
fr-GF: 0
fr-GN: 0
fr-GP: 0
fr-GQ: 0
fr-HT: 0
fr-KM: 0
fr-LU: 0
fr-MA: 0
fr-MC: 0
fr-MF: 0
fr-MG: 0
fr-ML: 0
fr-MQ: 0
fr-MR: 0
fr-MU: 0
fr-NC: 0
fr-NE: 0
fr-PF: 0
fr-PM: 0
fr-RE: 0
fr-RW: 0
fr-SC: 0
fr-SN: 0
fr-SY: 0
fr-TD: 0
fr-TG: 0
fr-TN: 0
fr-VU: 0
fr-WF: 0
fr-YT: 0
fur: 281
fy: 287
ga: 294
ga-GB: 0
gaa: 1
gd: 295
gl: 294
gsw: 276
gsw-FR: 0
gsw-LI: 0
gu: 294
guz: 222
gv: 2
ha: 294
ha-GH: 0
ha-NE: 0
haw: 20
he: 294
hi: 294
hi-Latn: 17
hnj: 1
hr: 294
hr-BA: 0
hsb: 294
ht: 0
hu: 295
hy: 294
ia: 295
id: 294
ie: 80
ig: 295
ii: 21
is: 294
it: 294
it-CH: 0
it-SM: 0
it-VA: 0
ja: 294
jgo: 82
jmc: 222
jv: 294
ka: 294
kaa: 279
kab: 222
kam: 222
kde: 222
kea: 293
kgp: 293
khq: 222
ki: 222
kk: 294
kk-KZ: 0
kkj: 1
kl: 1
kln: 222
km: 294
kn: 294
ko: 294
ko-CN: 0
ko-KP: 1
kok: 294
kok-Latn: 12
ks: 293
ks-Deva: 11
ksb: 221
ksf: 222
ksh: 287
ku: 295
kw: 1
kxv: 253
kxv-Deva: 253
kxv-Orya: 252
kxv-Telu: 253
ky: 294
lag: 222
lb: 288
lg: 222
lij: 3
lkt: 12
lld: 293
lmo: 1
ln: 229
ln-AO: 0
ln-CF: 0
ln-CG: 0
lo: 294
lrc: 23
lrc-IQ: 0
lt: 294
lu: 222
luo: 222
luy: 222
lv: 294
mai: 294
mas: 222
mas-TZ: 0
mer: 222
mfe: 222
mg: 222
mgh: 136
mgo: 2
mi: 294
mk: 294
ml: 294
mn: 294
mn-Mong-MN: 12
mni: 15
mr: 294
ms: 294
ms-BN: 0
ms-ID: 0
ms-SG: 0
mt: 285
mua: 222
mus: 1
my: 294
mzn: 287
naq: 222
nb: 0
nb-SJ: 0
nd: 222
nds: 2
nds-NL: 0
ne: 294
ne-IN: 0
nl: 295
nl-AW: 0
nl-BE: 0
nl-BQ: 0
nl-CW: 0
nl-SR: 0
nl-SX: 0
nmg: 222
nn: 45
nnh: 1
no: 295
nqo: 294
nso: 1
nus: 50
nyn: 222
oc: 3
oc-ES: 0
om: 294
om-KE: 0
or: 294
os: 18
os-RU: 0
osa: 1
pa: 294
pa-Arab: 1
pap: 1
pap-AW: 0
pcm: 294
pis: 1
pl: 294
prg: 2
ps: 294
ps-PK: 4
pt: 294
pt-AO: 0
pt-CH: 0
pt-CV: 0
pt-GQ: 0
pt-GW: 0
pt-LU: 0
pt-MO: 0
pt-MZ: 0
pt-PT: 70
pt-ST: 0
pt-TL: 0
qu: 294
qu-BO: 0
qu-EC: 0
quc: 1
raj: 2
rm: 293
rn: 222
ro: 294
ro-MD: 1
rof: 222
ru: 294
ru-BY: 0
ru-KG: 0
ru-KZ: 0
ru-MD: 0
ru-UA: 10
rw: 3
rwk: 222
sa: 11
sah: 25
saq: 222
sat: 11
sbp: 222
sc: 295
sd: 294
sd-Deva: 12
se: 283
se-FI: 20
se-SE: 0
seh: 221
ses: 222
sg: 222
shi: 222
shi-Latn: 222
shn: 2
shn-TH: 0
si: 294
sk: 294
sl: 294
smn: 251
sn: 221
so: 294
so-DJ: 0
so-ET: 0
so-KE: 0
sq: 294
sq-MK: 0
sq-XK: 0
sr: 294
sr-BA: 37
sr-ME: 11
sr-XK: 10
sr-Cyrl-ME: 11
sr-Latn: 294
sr-Latn-BA: 37
sr-Latn-XK: 10
ss: 1
ss-SZ: 0
st: 2
st-LS: 0
su: 12
sv: 294
sv-AX: 0
sv-FI: 0
sw: 294
sw-CD: 25
sw-KE: 80
sw-UG: 0
syr: 277
syr-SY: 0
szl: 1
ta: 294
ta-LK: 0
ta-MY: 0
ta-SG: 0
te: 294
teo: 222
teo-KE: 0
tg: 294
th: 294
ti: 295
ti-ER: 0
tk: 294
tn: 2
tn-BW: 0
to: 294
tok: 3
tpi: 7
tr: 294
tr-CY: 0
tt: 294
twq: 222
tzm: 222
ug: 288
uk: 294
ur: 294
ur-IN: 22
uz: 294
uz-Arab: 1
uz-Cyrl: 288
vai: 253
vai-Latn: 221
vec: 295
vi: 294
vmw: 1
vun: 222
wae: 282
wo: 294
xh: 294
xnr: 294
xog: 221
yav: 221
yi: 222
yo: 294
yo-BJ: 106
yrl: 293
yrl-CO: 9
yrl-VE: 9
yue: 294
yue-MO: 0
yue-Hans: 294
yue-Hant-CN: 0
za: 1
zgh: 225
zh: 294
zh-HK: 93
zh-MO: 93
zh-MY: 0
zh-SG: 0
zh-Hans-HK: 77
zh-Hans-MO: 77
zh-Hant: 294
zh-Hant-MY: 0
zu: 294

With script fallback:

af: 209
af-NA: 0
agq: 222
ak: 242
am: 0
ar: 0
ar-AE: 0
ar-BH: 0
ar-DJ: 0
ar-DZ: 0
ar-EG: 0
ar-EH: 0
ar-ER: 0
ar-IL: 0
ar-IQ: 0
ar-JO: 0
ar-KM: 0
ar-KW: 0
ar-LB: 0
ar-LY: 3
ar-MA: 0
ar-MR: 0
ar-OM: 0
ar-PS: 0
ar-QA: 0
ar-SA: 5
ar-SD: 0
ar-SO: 0
ar-SS: 0
ar-SY: 0
ar-TD: 0
ar-TN: 0
ar-YE: 0
as: 247
asa: 220
ast: 213
az: 264
az-IQ: 0
az-Arab: 1
az-Arab-TR: 0
az-Cyrl: 216
bal-Latn: 285
bas: 219
be: 253
be-tarask: 0
bem: 1
bez: 222
bg: 160
bgc: 1
bgn: 110
bgn-AE: 0
bgn-AF: 0
bgn-IR: 0
bgn-OM: 0
bho: 1
blo: 287
bm: 222
bn: 0
bn-IN: 4
bo: 12
bo-IN: 1
br: 187
brx: 258
bs: 245
bs-Cyrl: 232
bss: 1
ca: 206
ca-AD: 0
ca-ES-valencia: 0
ca-FR: 0
ca-IT: 0
ccp: 288
ccp-IN: 0
ce: 289
ceb: 15
cgg: 221
cho: 1
chr: 0
cic: 1
ckb: 291
ckb-IR: 0
cs: 231
csw: 10
cv: 160
cy: 158
da: 166
da-GL: 0
dav: 222
de: 187
de-AT: 1
de-BE: 0
de-CH: 7
de-IT: 0
de-LI: 0
de-LU: 0
dje: 221
doi: 4
dsb: 213
dua: 1
dyo: 108
dz: 286
ebu: 222
ee: 281
ee-TG: 0
el: 0
el-polyton: 0
el-CY: 0
en: 0
en-001: 9
en-150: 0
en-AE: 0
en-AG: 0
en-AI: 0
en-AS: 0
en-AT: 0
en-AU: 6
en-BB: 0
en-BE: 0
en-BI: 0
en-BM: 0
en-BS: 0
en-BW: 0
en-BZ: 0
en-CA: 21
en-CC: 0
en-CH: 0
en-CK: 0
en-CM: 0
en-CX: 0
en-CY: 0
en-CZ: 0
en-DE: 0
en-DG: 0
en-DK: 0
en-DM: 0
en-ER: 0
en-ES: 0
en-FI: 0
en-FJ: 0
en-FK: 0
en-FM: 0
en-FR: 0
en-GB: 0
en-GD: 0
en-GG: 0
en-GH: 0
en-GI: 0
en-GM: 0
en-GS: 0
en-GU: 0
en-GY: 0
en-HK: 0
en-HU: 0
en-ID: 0
en-IE: 0
en-IL: 0
en-IM: 0
en-IN: 0
en-IO: 0
en-IT: 0
en-JE: 0
en-JM: 0
en-KE: 0
en-KI: 0
en-KN: 0
en-KY: 0
en-LC: 0
en-LR: 0
en-LS: 0
en-MG: 0
en-MH: 0
en-MO: 0
en-MP: 0
en-MS: 0
en-MT: 0
en-MU: 0
en-MV: 0
en-MW: 0
en-MY: 0
en-NA: 0
en-NF: 0
en-NG: 0
en-NL: 0
en-NO: 0
en-NR: 0
en-NU: 0
en-NZ: 0
en-PG: 0
en-PH: 0
en-PK: 0
en-PL: 0
en-PN: 0
en-PR: 0
en-PT: 0
en-PW: 0
en-RO: 0
en-RW: 0
en-SB: 0
en-SC: 0
en-SD: 0
en-SE: 0
en-SG: 0
en-SH: 0
en-SI: 0
en-SK: 0
en-SL: 0
en-SS: 0
en-SX: 0
en-SZ: 0
en-TC: 0
en-TK: 0
en-TO: 0
en-TT: 0
en-TV: 0
en-TZ: 0
en-UG: 0
en-UM: 0
en-VC: 0
en-VG: 0
en-VI: 0
en-VU: 0
en-WS: 0
en-ZA: 0
en-ZM: 0
en-ZW: 0
en-Dsrt: 0
eo: 222
es: 205
es-419: 26
es-AR: 3
es-BO: 3
es-BR: 0
es-BZ: 0
es-CL: 4
es-CO: 3
es-CR: 3
es-CU: 0
es-DO: 3
es-EA: 0
es-EC: 3
es-GQ: 0
es-GT: 3
es-HN: 3
es-IC: 0
es-MX: 17
es-NI: 3
es-PA: 3
es-PE: 3
es-PH: 0
es-PR: 1
es-PY: 3
es-SV: 1
es-US: 18
es-UY: 0
es-VE: 3
et: 205
eu: 188
ewo: 222
fa: 285
fa-AF: 89
ff: 220
ff-BF: 0
ff-CM: 0
ff-GH: 0
ff-GM: 0
ff-GN: 0
ff-GW: 0
ff-LR: 0
ff-MR: 0
ff-NE: 0
ff-NG: 0
ff-SL: 0
ff-Adlm: 0
ff-Adlm-BF: 0
ff-Adlm-CM: 0
ff-Adlm-GH: 0
ff-Adlm-GM: 0
ff-Adlm-GW: 0
ff-Adlm-LR: 0
ff-Adlm-MR: 0
ff-Adlm-NE: 0
ff-Adlm-NG: 0
ff-Adlm-SL: 0
ff-Adlm-SN: 0
fi: 175
fil: 36
fo: 198
fo-DK: 0
fr: 217
fr-BE: 1
fr-BF: 0
fr-BI: 0
fr-BJ: 0
fr-BL: 0
fr-CA: 40
fr-CD: 0
fr-CF: 0
fr-CG: 0
fr-CH: 0
fr-CI: 0
fr-CM: 0
fr-DJ: 0
fr-DZ: 0
fr-GA: 0
fr-GF: 0
fr-GN: 0
fr-GP: 0
fr-GQ: 0
fr-HT: 0
fr-KM: 0
fr-LU: 0
fr-MA: 0
fr-MC: 0
fr-MF: 0
fr-MG: 0
fr-ML: 0
fr-MQ: 0
fr-MR: 0
fr-MU: 0
fr-NC: 0
fr-NE: 0
fr-PF: 0
fr-PM: 0
fr-RE: 0
fr-RW: 0
fr-SC: 0
fr-SN: 0
fr-SY: 0
fr-TD: 0
fr-TG: 0
fr-TN: 0
fr-VU: 0
fr-WF: 0
fr-YT: 0
fur: 281
fy: 189
ga: 281
ga-GB: 0
gaa: 0
gd: 270
gl: 201
gsw: 276
gsw-FR: 0
gsw-LI: 0
gu: 0
guz: 222
gv: 2
ha: 251
ha-GH: 0
ha-NE: 0
haw: 20
he: 0
hi: 0
hi-Latn: 17
hnj: 1
hr: 238
hr-BA: 0
hsb: 212
ht: 77
hu: 224
hy: 0
ia: 149
id: 164
ie: 49
ig: 53
ii: 0
is: 271
it: 167
it-CH: 0
it-SM: 0
it-VA: 0
ja: 0
jgo: 82
jmc: 222
jv: 219
ka: 0
kaa: 279
kab: 222
kam: 222
kde: 222
kea: 261
kgp: 290
khq: 222
ki: 222
kk: 120
kk-KZ: 0
kkj: 1
kl: 1
kln: 222
km: 0
kn: 0
ko: 0
ko-CN: 0
ko-KP: 1
kok: 210
kok-Latn: 12
ks: 291
ks-Deva: 6
ksb: 221
ksf: 222
ksh: 287
ku: 254
kw: 1
kxv: 253
kxv-Deva: 100
kxv-Orya: 208
kxv-Telu: 101
ky: 117
lag: 222
lb: 187
lg: 222
lij: 3
lkt: 12
lld: 293
lmo: 1
ln: 229
ln-AO: 0
ln-CF: 0
ln-CG: 0
lo: 0
lrc: 23
lrc-IQ: 0
lt: 277
lu: 222
luo: 222
luy: 222
lv: 275
mai: 128
mas: 222
mas-TZ: 0
mer: 222
mfe: 222
mg: 222
mgh: 136
mgo: 2
mi: 288
mk: 230
ml: 0
mn: 201
mn-Mong-MN: 12
mni: 12
mr: 208
ms: 113
ms-BN: 0
ms-ID: 0
ms-SG: 0
mt: 241
mua: 222
mus: 1
my: 0
mzn: 287
naq: 222
nb: 0
nb-SJ: 0
nd: 222
nds: 2
nds-NL: 0
ne: 228
ne-IN: 0
nl: 185
nl-AW: 0
nl-BE: 0
nl-BQ: 0
nl-CW: 0
nl-SR: 0
nl-SX: 0
nmg: 222
nn: 45
nnh: 1
no: 150
nqo: 294
nso: 1
nus: 50
nyn: 222
oc: 3
oc-ES: 0
om: 287
om-KE: 0
or: 0
os: 18
os-RU: 0
osa: 1
pa: 0
pa-Arab: 1
pap: 1
pap-AW: 0
pcm: 294
pis: 1
pl: 216
prg: 2
ps: 285
ps-PK: 4
pt: 232
pt-AO: 0
pt-CH: 0
pt-CV: 0
pt-GQ: 0
pt-GW: 0
pt-LU: 0
pt-MO: 0
pt-MZ: 0
pt-PT: 70
pt-ST: 0
pt-TL: 0
qu: 200
qu-BO: 0
qu-EC: 0
quc: 1
raj: 1
rm: 173
rn: 222
ro: 163
ro-MD: 1
rof: 222
ru: 0
ru-BY: 0
ru-KG: 0
ru-KZ: 0
ru-MD: 0
ru-UA: 10
rw: 2
rwk: 222
sa: 11
sah: 16
saq: 222
sat: 0
sbp: 222
sc: 220
sd: 283
sd-Deva: 4
se: 283
se-FI: 20
se-SE: 0
seh: 221
ses: 222
sg: 222
shi: 222
shi-Latn: 222
shn: 2
shn-TH: 0
si: 0
sk: 229
sl: 239
smn: 251
sn: 221
so: 268
so-DJ: 0
so-ET: 0
so-KE: 0
sq: 257
sq-MK: 0
sq-XK: 0
sr: 232
sr-BA: 37
sr-ME: 11
sr-XK: 10
sr-Cyrl-ME: 11
sr-Latn: 250
sr-Latn-BA: 37
sr-Latn-XK: 10
ss: 1
ss-SZ: 0
st: 1
st-LS: 0
su: 10
sv: 179
sv-AX: 0
sv-FI: 0
sw: 163
sw-CD: 25
sw-KE: 80
sw-UG: 0
syr: 0
syr-SY: 0
szl: 1
ta: 0
ta-LK: 0
ta-MY: 0
ta-SG: 0
te: 0
teo: 222
teo-KE: 0
tg: 165
th: 0
ti: 207
ti-ER: 0
tk: 255
tn: 1
tn-BW: 0
to: 289
tok: 3
tpi: 7
tr: 226
tr-CY: 0
tt: 122
twq: 222
tzm: 222
ug: 286
uk: 221
ur: 287
ur-IN: 22
uz: 254
uz-Arab: 1
uz-Cyrl: 133
vai: 253
vai-Latn: 216
vec: 222
vi: 123
vmw: 1
vun: 222
wae: 282
wo: 274
xh: 294
xnr: 56
xog: 221
yav: 221
yi: 222
yo: 276
yo-BJ: 106
yrl: 288
yrl-CO: 9
yrl-VE: 9
yue: 33
yue-MO: 0
yue-Hans: 136
yue-Hant-CN: 0
za: 1
zgh: 225
zh: 0
zh-HK: 93
zh-MO: 93
zh-MY: 0
zh-SG: 0
zh-Hans-HK: 77
zh-Hans-MO: 77
zh-Hant: 0
zh-Hant-MY: 0
zu: 291
und-Adlm: 293
und-Arab: 294
und-Armn: 294
und-Beng: 294
und-Cans: 0
und-Cher: 294
und-Cyrl: 294
und-Cyrl-tarask: 0
und-Deva: 294
und-Dsrt: 273
und-Ethi: 294
und-Geor: 294
und-Grek: 294
und-Grek-polyton: 294
und-Gujr: 294
und-Guru: 294
und-Hans: 294
und-Hant: 294
und-Hebr: 294
und-Jpan: 294
und-Khmr: 294
und-Knda: 294
und-Kore: 294
und-Laoo: 294
und-Latn: 295
und-Latn-valencia: 0
und-Mlym: 294
und-Mong: 0
und-Mymr: 294
und-Nkoo: 0
und-Olck: 11
und-Orya: 294
und-Sinh: 294
und-Syrc: 277
und-Taml: 294
und-Telu: 294
und-Thai: 294
und-Yiii: 21

sffc added 4 commits September 4, 2025 21:39
@sffc
UnvalidatedTinyAsciiStr::as_potential_utf8
47bf4dd
@sffc
Initial commit: dnametest.rs
ec1899e
@sffc
Add script fallback
bf3388f
@sffc
Measure the zerotrie sizes:
bc59654 
locales_only_zerotrie: 3339
regions_only_zerotrie: 1268
sparse_map: 50751
sparse_zerotrie: 272897
hybrid_sparse_map: 2223
num_dense_locales: 207 (122130 B)
hybrid_sparse_zerotrie: 13146
@sffc
Copy link
Member Author

sffc commented Sep 5, 2025 •
edited
Loading

Some sizes based on CLDR 46* data for region display names:

Structure What it Contains Size in Bytes
ZeroTrie Locales 1697
ZeroTrie Regions 1268
ZeroTrie Locales + Sparse Regions 272897
ZeroTrie Locales + Hybrid Sparse Regions 13146
Matrix Dense Regions 193520
Matrix Hybrid Dense Regions 122130

What this means for the overall storage requirements:

Strategy Requirements Total Size
Sparse Single sparse zerotrie 272 kB
Dense Dense matrix, locales trie, regions trie 196 kB
Hybrid Hybrid locales trie, hybrid matrix, regions trie 137 kB

I could implement the hybrid approach either in datagen or in zerotrie itself. If I did it in zerotrie, I might call it a "suffix optimization". Not 100% sure the heuristic I would end up using for it.

Reminder: this is only for the lookup table (locale/region to string index).

@sffc sffc mentioned this pull request Sep 5, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@robertbastian robertbastian Awaiting requested review from robertbastian robertbastian will be requested when the pull request is marked ready for review robertbastian is a code owner

@zbraniecki zbraniecki Awaiting requested review from zbraniecki zbraniecki will be requested when the pull request is marked ready for review zbraniecki is a code owner

@Manishearth Manishearth Awaiting requested review from Manishearth Manishearth will be requested when the pull request is marked ready for review Manishearth is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sffc