Skip to content

Prefer IndexUnicodeProperties over ToolUnicodePropertySource #488

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 28, 2023

Conversation

eggrobin
Copy link
Member

@eggrobin eggrobin commented May 26, 2023

See discussion in #432 (comment):
@eggrobin:

[…] the question is whether this should be inverted.

        propSource = getProperties(version);  // ToolUnicodePropertySource.
        property = propSource.getProperty(xPropertyName); 
        if (property == null) { 
            propSource = getIndexedProperties(version);   // IndexUnicodeProperties.
            property = propSource.getProperty(xPropertyName); 

@macchiati:

I agree, it should be inverted. There are a few constructed properties
(non-UCD) that are only available via Tool... That is, the extra
normalization properties. So those should fallback to the Tool

This uses IndexUnicodeProperties except for the following, where ToolUnicodePropertySource is used:

  1. the toNF[ ] pseudoproperties;
  2. properties that did not exist in that version of Unicode, if the version is requested with an R (for Retroactive) rather than a U.

This is used for the invariant about NFKC_SCF, which does not exist until 15.1, but whose derivation would be stable very far back.

This also fixes defaults (incorrectly null instead of <code point>) in the NFKC_CF and NFKC_SCF returned by ToolUnicodePropertySource.

@eggrobin eggrobin changed the title Prefer Prefer IndexUnicodeProperties over ToolUnicodePropertySource May 26, 2023
@eggrobin
Copy link
Member Author

This also fixes defaults (incorrectly null instead of <code point>) in the NFKC_CF and NFKC_SCF returned by ToolUnicodePropertySource.

Ah, the code that generates DerivedNormalizationProps was relying on those nulls. I will have to fix that.

@eggrobin
Copy link
Member Author

TestTestUnicodeInvariants.testUnicodeInvariants:35 TestUnicodeInvariants.testInvariants() failed ==> expected: <0> but was: <2>

That’s odd: on my machine it works.

Copy link
Member

@macchiati macchiati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work

@eggrobin eggrobin marked this pull request as ready for review May 28, 2023 20:52
@eggrobin eggrobin merged commit df37d9d into unicode-org:main May 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants