Skip to content

Implementing namespacing #524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Nov 29, 2023
Merged

Implementing namespacing #524

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
d850cf4
Implementing namespacing
aphillips Nov 13, 2023
1cf7f39
Update syntax.md
aphillips Nov 13, 2023
30cdac6
Update formatting.md
aphillips Nov 14, 2023
d875dd3
Fix one instance of normative language
aphillips Nov 14, 2023
a75908a
Update spec/formatting.md
aphillips Nov 14, 2023
7f099a2
Update spec/formatting.md
aphillips Nov 14, 2023
bcff52e
Update message.abnf
aphillips Nov 14, 2023
7e73ab7
Adjust to match syntax changes
aphillips Nov 14, 2023
732ab83
Fixing list numbering and name-handling wording
aphillips Nov 14, 2023
d69a927
Update spec/formatting.md
aphillips Nov 15, 2023
1dc97bf
Update spec/formatting.md
aphillips Nov 15, 2023
85d7ead
Update spec/formatting.md
aphillips Nov 15, 2023
c3ad9d9
Update spec/formatting.md
aphillips Nov 15, 2023
9e40a9c
Remove `:` from `name-char`
aphillips Nov 15, 2023
233c217
Clean up references to NCName and clarify no colons
aphillips Nov 20, 2023
9830785
Update formatting based on comments
aphillips Nov 20, 2023
341717c
Update spec/formatting.md
aphillips Nov 20, 2023
290a7c4
Address comments
aphillips Nov 20, 2023
2c471a5
Address part of @eemeli's feedback
aphillips Nov 25, 2023
4507a4e
Update syntax.md
aphillips Nov 25, 2023
6b15f10
Update spec/message.abnf
aphillips Nov 26, 2023
cccb7ad
Update spec/formatting.md
aphillips Nov 26, 2023
ef83110
Update spec/formatting.md
aphillips Nov 26, 2023
e9fb373
Update spec/formatting.md
aphillips Nov 26, 2023
17d714e
Update spec/syntax.md
aphillips Nov 27, 2023
714b4fe
Address naming of `identifier` and `name`
aphillips Nov 27, 2023
0c50ab7
Implementing `identifier` and `name` in syntax.md
aphillips Nov 27, 2023
3625ac4
Implement `identifier` vs. `name` in formatting.md
aphillips Nov 27, 2023
94dfbd4
Update spec/syntax.md
aphillips Nov 27, 2023
4692f72
Changes per 2023-11-27 call
aphillips Nov 27, 2023
3adb4aa
Update message.abnf
aphillips Nov 27, 2023
0b7042b
Add `:` to unquoted
aphillips Nov 27, 2023
a4920fa
Update spec/formatting.md
aphillips Nov 28, 2023
d3e0e58
Update spec/message.abnf
aphillips Nov 28, 2023
938d7d5
Apply suggestions from code review
aphillips Nov 28, 2023
fd62645
Clean formatting
aphillips Nov 28, 2023
60a1740
Address double-emission of unresolved variable
aphillips Nov 28, 2023
023b5a8
Update spec/formatting.md
aphillips Nov 28, 2023
1e9e644
Remove extra MUSTard
aphillips Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 30 additions & 14 deletions spec/formatting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,17 +197,28 @@ the following steps are taken:

1. If the _expression_ includes an _operand_, resolve its value.
If this fails, use a _fallback value_ for the _expression_.
2. Based on the _function_ starting sigil and _name_,
find the appropriate function implementation from the _function registry_.
If the registry does not define an implementation for this _name_,
2. Resolve the _identifier_ of the _function_ and, based on the starting sigil,
find the appropriate function implementation to call.
If the implementation cannot find the function,
or if the _identifier_ includes a _namespace_ that the implementation does not support,
emit an Unknown Function error
and use a _fallback value_ for the _expression_.
3. Resolve the _option_ values to a mapping of string identifiers to values.

Implementations are not required to implement _namespaces_ or installable
_function registries_.
Comment on lines +206 to +208
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems out of place. If we do want to mention something like this, it should not be here, in the middle of an algorithmic sequence of steps.

Suggested change
Implementations are not required to implement _namespaces_ or installable
_function registries_.


3. Resolve the _options_ to a mapping of string identifiers to values.
If _options_ is missing, the mapping will be empty.
For each _option_:
- If its right-hand side successfully resolves to a value,
bind the _name_ of the _option_ to the resolved value in the mapping.
- Otherwise, do not bind the _name_ of the _option_ to any value in the mapping.
4. Call the function implementation with the following arguments:
- Resolve the _identifier_ of the _option_.
- If the _option_'s _identifier_ already exists in the resolved mapping of _options_,
emit a Duplicate Option Name error.
- If the _option_'s right-hand side successfully resolves to a value,
bind the _identifier_ of the _option_ to the resolved value in the mapping.
- Otherwise, bind the _identifier_ of the _option_ to an unresolved value in the mapping.
(Note that an Unresolved Variable error will have been emitted.)
4. Remove from the resolved mapping of _options_ any binding for which the value is an unresolved value.
5. Call the function implementation with the following arguments:

- The current _locale_.
- The resolved mapping of _options_.
Expand All @@ -219,15 +230,20 @@ the following steps are taken:
as long as reasonable precautions are taken to keep the function interface
simple and minimal, and avoid introducing potential security vulnerabilities.

As implementations MAY allow custom functions to be defined by users,
their access to the _formatting context_ SHOULD be minimal and read-only,
and their execution time SHOULD be limited.
An implementation MAY define its own functions.
An implementation MAY allow custom functions to be defined by users.

Function access to the _formatting context_ MUST be minimal and read-only,
and execution time SHOULD be limited.

Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "implementation-defined" include the implementations of the default registry's functions? Or do you mean other functions built into the implementation, but not present in the default registry?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking of going back to the design document and adding some use cases to illustrate these. In this case, I mean "non-default registry functions". I will add text to clarify.

Suppose you're Mihai and you're creating the ICU4J implementation. The :datetime function in the default registry might be backed by com.ibm.icu.text.DateFormat as an implementing class. There's no need to prefix :datetime for that. That's just implementation.

Mihai might want to expose skeleton though. That's not (currently) a default registry option. So he'd prefix it as icu:skeleton in his implementation.

Suppose he then goes on to implement com.ibm.icu.text.DateIntervalFormat as a function. That's not in the default registry, so it gets :icu:dateinterval and its options might or might not be namespaced.

Suppose Mihai goes on to implement an SPI for custom functions. Suppose I write my own date formatter. It doesn't replace the one in :dateformat, so it gets a prefix like :app:dateformat. The standard options might not be prefixed, but my custom options would be. ICU's options would probably be unrecognized to me. So you could see expressions like:

{$now :datetime icu:skeleton=yMMMd}
{$now :icu:dateinterval end=$then skeleton=yMMM}
{$now :app:datetime dateStyle=short :app:wonderfulness=high}

Suppose you're a tool implementer working with messages written for Mihai's implementation with my add-on. When you see a namespace prefix, that tells you that the function or the option is non-standard. The prefix might tell you where to look for the add-on registry or at least be something that you can key locally. Tool consumption is a key reason for namespacing.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding these use-cases in form of such user journeys sounds great. I'll be happy to contribute. Open a new PR?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be best to add a subsection to "Function Resolution" that collects these directions about what's possible and recommended for implementations and users to do with functions?

Suggested change
Implementation-defined _functions_ SHOULD use an implementation-defined _namespace_.


5. If the call succeeds,
resolve the value of the _expression_ as the result of that function call.
If the call fails or does not return a valid value,
emit a Resolution error and use a _fallback value_ for the _expression_.


### Fallback Resolution

A **_fallback value_** is the resolved value emitted when an _expression_ cannot be resolved.
Expand Down Expand Up @@ -255,13 +271,13 @@ The _fallback value_ depends on the contents of the _expression_:
> Example: `$user`

- _expression_ with no _operand_:
the _function_ starting sigil followed by its _name_
the _function_ starting sigil followed by its _identifier_

> Examples: `:platform`, `+tag`, `-tag`

- Otherwise: The U+FFFD REPLACEMENT CHARACTER `�` character

_Option_ names and values are not included in the _fallback value_.
_Option_ identifiers and values are not included in the _fallback value_.

When an error occurs in an _expression_ with a _variable_ _operand_
and the _variable_ refers to a local _declaration_,
Expand Down Expand Up @@ -817,7 +833,7 @@ These are divided into the following categories:
> }}
> ```

- A **Duplicate Option Name error** occurs when the same _name_
- A **Duplicate Option Name error** occurs when the same _identifier_
appears on the left-hand side
of more than one _option_ in the same _expression_.

Expand Down
19 changes: 9 additions & 10 deletions spec/message.abnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ annotation = (function *(s option)) / reserved / private-use

literal = quoted / unquoted
variable = "$" name
function = (":" / "+" / "-") name
option = name [s] "=" [s] (literal / variable)
function = (":" / "+" / "-") identifier
option = identifier [s] "=" [s] (literal / variable)

; reserved keywords are always lowercase
input = %s"input"
Expand All @@ -45,13 +45,10 @@ quoted-char = %x0-5B ; omit \
/ %x7D-D7FF ; omit surrogates
/ %xE000-10FFFF

; based on https://www.w3.org/TR/xml/#NT-Nmtoken,
; but cannot start with U+002D HYPHEN-MINUS or U+003A COLON ":"
unquoted = unquoted-start *name-char
unquoted = unquoted-start *(name-char / ":")
unquoted-start = name-start / DIGIT / "."
/ %xB7 / %x300-36F / %x203F-2040


; reserve sigils for private-use by implementations
private-use = private-start reserved-body
private-start = "^" / "&"
Expand All @@ -70,15 +67,17 @@ reserved-char = %x00-08 ; omit HTAB and LF
/ %x7E-D7FF ; omit surrogates
/ %xE000-10FFFF

; based on https://www.w3.org/TR/xml/#NT-Name,
; but cannot start with U+003A COLON ":"
name = name-start *name-char
; identifier matches https://www.w3.org/TR/REC-xml-names/#NT-QName
; name matches https://www.w3.org/TR/REC-xml-names/#NT-NCName
identifier = [namespace ":"] name
namespace = name
name = name-start *name-char
name-start = ALPHA / "_"
/ %xC0-D6 / %xD8-F6 / %xF8-2FF
/ %x370-37D / %x37F-1FFF / %x200C-200D
/ %x2070-218F / %x2C00-2FEF / %x3001-D7FF
/ %xF900-FDCF / %xFDF0-FFFD / %x10000-EFFFF
name-char = name-start / DIGIT / "-" / "." / ":"
name-char = name-start / DIGIT / "-" / "."
/ %xB7 / %x300-36F / %x203F-2040

text-escape = backslash ( backslash / "{" / "}" )
Expand Down
77 changes: 58 additions & 19 deletions spec/syntax.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ external input value does not appear in a _declaration_.
> [!Note]
> These restrictions only apply to _declarations_.
> A _placeholder_ or _selector_ can apply a different annotation to a _variable_
> than one applied to the same _variable_ name in a _declaration_.
> than one applied to the same _variable_ named in a _declaration_.
> For example, this message is _valid_:
> ```
> {{
Expand Down Expand Up @@ -482,7 +482,7 @@ and vice versa.
> {+button}Submit{-button} or {+link}cancel{-link}.
> ```

A _function_ consists of a prefix sigil followed by a _name_.
A _function_ consists of a prefix sigil followed by an _identifier_.
The following sigils are used for _functions_:

- `:` for a _standalone_ function
Expand All @@ -497,16 +497,16 @@ _Options_ are not required.
An **_<dfn>option</dfn>_** is a key-value pair
containing a named argument that is passed to a _function_.

An _option_ has a _name_ and a _value_.
The _name_ is separated from the _value_ by an U+003D EQUALS SIGN `=` along with
An _option_ has an _identifier_ and a _value_.
The _identifier_ is separated from the _value_ by an U+003D EQUALS SIGN `=` along with
optional whitespace.
The value of an _option_ can be either a _literal_ or a _variable_.

Multiple _options_ are permitted in an _annotation_.
Each _option_ is separated by whitespace.

```abnf
option = name [s] "=" [s] (literal / variable)
option = identifier [s] "=" [s] (literal / variable)
```

> Examples of _functions_ with _options_
Expand Down Expand Up @@ -672,21 +672,65 @@ unquoted-start = name-start / DIGIT / "."
/ %xB7 / %x300-36F / %x203F-2040
```

### Names
### Names and Identifiers

A **_<dfn>name</dfn>_** is an identifier for a _variable_ (prefixed with `$`),
An **_<dfn>identifier</dfn>_** is a character sequence that
identifies a _function_ or _option_.
Each _identifier_ consists of a _name_ optionally preceeded by
a _namespace_.
When present, the _namespace_ is separated from the _name_ by a
U+003A COLON `:`.
Built-in _functions_ and their _options_ do not have a _namespace_ identifier.

_Function_ _identifiers_ are prefixed with `:`, `+`, or `-`.
_Option_ _identifiers_ have no prefix.

A **_<dfn>name</dfn>_** is a character sequence used in an _identifier_
or as the name for for a _variable_.

_Variable_ names are prefixed with `$`.
for a _function_ (prefixed with `:`, `+` or `-`),
or for an _option_ (these have no prefix).
The namespace for _names_ is based on XML's [Name](https://www.w3.org/TR/xml/#NT-Name),
with the restriction that it MUST NOT start with `:`,
as that would conflict with the _function_ start character.
Otherwise, the set of characters allowed in names is large.

Valid content for _names_ is based on <cite>Namespaces in XML 1.0</cite>'s
[NCName](https://www.w3.org/TR/xml-names/#NT-NCName).
This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name)
in that it MUST NOT contain a U+003A COLON `:`.
Otherwise, the set of characters allowed in a _name_ is large.

> [!NOTE]
> _External variables_ can be passed in that are not valid _names_.
> Such variables cannot be referenced in a _message_,
> but are not otherwise errors.

Examples:
> A variable:
>```
>This has a {$variable}
>```
>A function:
> ```
> This has a {:function}
> ```
> An add-on function from the `icu` namespace:
> ```
> This has a {:icu:function}
> ```
> An option and an add-on option:
> ```
> This has {:options option=value icu:option=add_on}
> ```

Support for _namespaces_ and their interpretation is implementation-defined
in this release.

```abnf
variable = "$" name
function = (":" / "+" / "-") name
function = (":" / "+" / "-") identifier
option = identifier [s] "=" [s] (literal / variable)

name = name-start *name-char
identifier = [namespace ":"] name
namespace = name
name = name-start *name-char
name-start = ALPHA / "_"
/ %xC0-D6 / %xD8-F6 / %xF8-2FF
/ %x370-37D / %x37F-1FFF / %x200C-200D
Expand All @@ -696,11 +740,6 @@ name-char = name-start / DIGIT / "-" / "." / ":"
/ %xB7 / %x300-36F / %x203F-2040
```

> [!NOTE]
> _External variables_ can be passed in that are not valid _names_.
> Such variables cannot be referenced in a _message_,
> but are not otherwise errors.

### Escape Sequences

An **_<dfn>escape sequence</dfn>_** is a two-character sequence starting with
Expand Down