Do we have a place to record conformance tests? This requires rejection of inputs that naïve implementations might accept, e.g.

• | 1|
• |.1|
• |1.|
• |01|
• |+1|
• |0x1|

Yes, we do! See the folder 'test' in our repo. I have filed an issue to capture this comment.

This arrangement seems odd to me; I would expect options for :number to be defined in the same section as the function itself rather than in a section introducing both :number and :integer.

Your suggested organization of the sections is superior to what I have here. Once all other comments are cleared, I will reorganize that way.

I'm not convinced that this note is helpful; the same dynamic default behavior is also true of numberingSystem.

Could we instead put something above all of the options in a non-NOTE style that says something like:

Some options do not have default values defined in this specification.
The defaults for these options are implementation-dependent.
In general, these types of defaults depend on the locale, the value of other options,
or both.

I implemented this. See what you think.

We should clarify which functions take which of these values. All can't be valid for all, as e.g. an XML time is insufficient input for :date.

Also, are any timezones in literal inputs used in the formatted outputs? For example, when formatting

{12:34+06:00 :time}

is the resulting formatted time in the system timezone, or in +06?

How about UTC, what timezone does this render in?

{12:34Z :time}

As we've left out timeZone from the available options, also leaving out timezone parsing would be one way to resolve this.

Oh boy, where do I start? 😈

The values can parse into various kinds of date/time objects. Many (most?) of these can be formatted by any of the functions given here, although the results may be not be very useful. Specifically, a time formatted as a date or a date formatted as a time is probably not very interesting.

XMLSchema date/time values do not have timezones. They only support zone offsets. The only zone offset that can be 100% certainly made into a time zone is Z because that one means "UTC".

The time zone is very important in rendering date and time values. I argued at length in the previous PR (and tried to make the point in the call) that we should not excommunicate timezone as an option to "future". Without that option, there are only the following options for managing time zone in formatting:

• the implementation uses the system default
• the implementation provides an API for setting the time zone of an entire message
  (this is not a behavior we specify because it is beyond our remit, although probably
  we should point it out)
• the implementation assumes UTC

In addition, date/time literals with no offset should be handled as floating time values (even if the implementation uses some specific time zone, the value should be formatted so that the field values don't change--often this is done using UTC as the time zone, ignoring local settings)

Note well that "time zone parsing" doesn't exist here. The zone offset (usually include Z) parses into the underlying incremental ("instant") value. That is, all of the following are the same time value:

• 2007-01-01T01:00:00.000-01:00
• 2007-01-01T01:00:00.000Z
• 2007-01-01T01:00:00.000+01:00
• 2006-12-31T23:00:00.000-01:00

Literals missing one part should be permitted to treat that part as 0. Some examples:

If we didn't do this, users used to working with partial values would have messages fail instead of doing what their code usually does. The group will need to decide if we bring timeZone back in Tech Preview. We should also discuss whether to specify any of this behavior.

Some of what I express above might be controversial, in that we could make different decisions about forcing floating time behavior or permitting partial values.

Note to @eemeli... see disconnected conversation below ⬇️

I see what you're doing!

The name or number-literal productions are narrower than the first description (which is the guts of a quoted literal). That narrower production defines what's allowed in an unquoted literal, which has no real bearing here (it's a detail of the syntax).

The verb "generated" also bothers me, as one doesn't generate the operand. I kind of want to say the "resolved value of a literal", since one of the things that should happen is that the quoted-escape becomes unescaped in the data-model or in-memory representation. It's only escaped in the syntax to avoid conflicting with the grammar.

Perhaps:

We should add an example too.

I rewrote and added an example. Thanks!

I think the typical processing model is:

1. Given the string form of the literal embedded in the message.
2. We fix any features that were used to make the literal fit the syntax: quotes, or quoted-escape sequences, so that we have it in a clean string.
3. We convert into the internal datatype (eg BigDecimal, or Datetime) expected by the implementation's version of the function (eg :number or :datetime).
4. We convert the accumulated options for the function also into the internal datatypes expected by the function.
5. We pass everything to the function, plus an enum value specifying whether we want a string or a data-model part
6. We get the result and do something interesting with it (insert it into a string buffer that we are building, or into a data model

The problem with "resolved value" is that some people might think of it as #2, and some people might think of it as #3. Maybe we can have a special term for just #2?

@macchiati That specific "typical" model_does not work for MessageFormat. Specifically, MF cannot convert the value to any kind of datatype. That is the responsibility of the function. This is because MF itself is typeless.

This does not mean that typed values passed in as arguments "lose their shape". Implementations can (and are expected) to do reflection or type checking upon them. But only string literal values exist inside MF2 syntax (you can use a variable to refer to a typed argument value).

This is why this specific PR talks about the literal form of the operand and then various "implementation-defined types". The expression {|2024-02-17| :datetime} is converted to a date inside the :datetime function. This is important because it allows types that MessageFormat does not know about itself to be supported via plug-in functions.

The term for 2 is "the resolved value of the literal". (More elsewhere on the problem of 'resolved value').

Note that "resolved" deals with transitive assignment cases:

.local $b = {$a}
.local $c = {$b}

... to make the "resolved value" of $c be the value contained inside of $a (plus whatever happened along the way, e.g. if the first line were .local $b = {$a :transform operation=uppercase} you might expect $c to be uppercase.).

In this specific case, we are talking about the :string function and we're talking about passing it a string. What concrete change is needed?

I was confused as to what you meant by a 'resolved value', and could see two possible interpretations. My comment was specific to :string (because as you say the results are always string comparisons anyway).

The spec always has to work in terms of strings, because it knows nothing about datatypes, and can't require particular datatypes because they vary by environment and implementation capabilities.

However, I would expect many MF2 implementations to actually operate with inputs that are not strings, but rather datatypes: both primitive and non-primitive. We can't have a spec that precludes implementations of MF2 from doing that. And those implementations will want to keep the values in datatypes where feasible, instead of always stringifying.

So, for example, it should be possible for an implementation to compile a MF2 into a data structure whereby when it knows that $input is a long, it can convert a literal for matching also into a long in that data structure, and just do long comparison.

Similarly, for plural matching, the compiled data structure could contain an enum value `few', instead of converting the an internal value for the plural category into a string and then doing a string comparison.

I think the typical processing model is:

1. Given the string form of the literal embedded in the message.
2. We fix any features that were used to make the literal fit the syntax: quotes, or quoted-escape sequences, so that we have it in a clean string.
3. We convert into the internal datatype (eg BigDecimal, or Datetime) expected by the implementation's version of the function (eg :number or :datetime).
4. We convert the accumulated options for the function also into the internal datatypes expected by the function.
5. We pass everything to the function, plus an enum value specifying whether we want a string or a data-model part
6. We get the result and do something interesting with it (insert it into a string buffer that we are building, or into a data model

The problem with "resolved value" is that some people might think of it as #2, and some people might think of it as #3. Maybe we can have a special term for just #2?

Not sure what this means; wouldn't we just treat a char as a string with a single code point?

That's exactly what I mean: don't overlook char and java.lang.Character

This is not clear. If the key is |1| and the operand is "1", they wouldn't match. Is there some place that specifies that at this point in the algorithm the key has had its quotes removed and been unescaped?

This is dangerous. The message is not in NFC and the operand is; do we really want that to fail?

I think at a minimum we should recommend that messages be in NFC.

https://www.w3.org/TR/charmod-norm/

Requiring implementations to do normalization can be a big ask (I spent a long time at W3C asking...) and there are counterarguments. We should certainly recommend that users stick to normalized sequences. That is what this note is trying to do. We can mention NFC here if you think that would help.

Yes, I think that would be useful. (I wasn't proposing requiring it.)

I would also appreciate a recommendation for NFC normalization specifically for this comparison.

I would also appreciate a recommendation for NFC normalization specifically for this comparison.

If you mean requiring or suggesting that implementations normalize the key and value when performing the comparison, it is important that implementations NOT do that. That would make implementations work differently depending on whether they normalize or not.

Disagree: 'onto' does not convey 'at the end of', and the goal is to prioritize these below the literal matches.

I can't find anywhere where we define what "resolved value" means.

The term "resolved value" lacks a formal definition, but the various sorts of resolved values are defined here (in the formatting spec). This may be a gap.

I will do a PR with a formal definition because you're not the first to ask.

Sounds good.

Nitpick:
s / needs to be quoted / MUST be quoted /
?

This is a non-normative note, so using MUST would be out of place.

Hm? Doesn't $example contain the locale-dependent string -1,234.567 which doesn't conform to the grammar?

No, because $example is a resolved rather than a formatted value. So it's not a formatted string, but some implementation-defined value which when formatted to a string would format as e.g. -1,234.567, but it also contains the numerical value 1234.567.

I'm still not super comfortable with the decision of MF2 to name options without defining how they work, but the compromise I'm okay with is that things that have semantic meaning should be well specified, and things that are formatting hints can be left implementation-defined. We discussed this point at length in the extended MF meeting last Wednesday and decided to cut number formatting options that carry semantic meaning, such as currency and unit, until we can discuss their design further.