Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICU-22747 MF2: Validate values of integer-valued options for :number and :integer #2973

Draft
wants to merge 1 commit into
base: maint/maint-75
Choose a base branch
from

Conversation

catamorphism
Copy link
Contributor

@catamorphism catamorphism commented Apr 18, 2024

Previously, it wasn't an error to write a message like::

.local $foo = {1 :integer minimumIntegerDigits=foo} {{$foo}}

This is an error according to the spec, because the minimumIntegerDigits option to :integer must have a value that's a non-negative integer.

This change adds validation of options that must be integer-valued to the implementations of the built-in :integer and :number functions. Other options (most of which have small sets of string values that are allowed as valid options) are not validated yet.

Checklist
  • Required: Issue filed: https://unicode-org.atlassian.net/browse/ICU-22747
  • Required: The PR title must be prefixed with a JIRA Issue number.
  • Required: The PR description must include the link to the Jira Issue, for example by completing the URL in the first checklist item
  • Required: Each commit message must be prefixed with a JIRA Issue number.
  • Issue accepted (done by Technical Committee after discussion)
  • Tests included, if applicable
  • API docs and/or User Guide docs changed or added, if applicable

…and :integer

Previously, it wasn't an error to write a message like::

.local $foo = {1 :integer minimumIntegerDigits=foo} {{$foo}}

This is an error according to the spec, because the `minimumIntegerDigits`
option to `:integer` must have a value that's a non-negative integer.

This change adds validation of options that must be integer-valued
to the implementations of the built-in `:integer` and `:number` functions.
Other options (most of which have small sets of string values that are
allowed as valid options) are not validated yet.
@aphillips
Copy link
Member

Previously, it wasn't an error to write a message like::

.local $foo = {1 :integer minimumIntegerDigits=foo} {{$foo}}

This is an error according to the spec, because the minimumIntegerDigits option to :integer must have a value that's a non-negative integer.

It is not an error to write a message like that, from the point of view that this is a "valid" message.

I carefully searched the formatting and registry sections and double-checked errors. It is not currently required to be an error (as an alternative, an implementation might ignore the option, although I'm dubious that this is a Good Idea). The registry part of the spec says what the value space for digit options is, but not what to do if the value provided doesn't match it. (It also fails to clearly state that what to do is implementation defined). It is permitted that implementations emit errors during function resolution (in which case, this will be an Invalid Expression error). When this is addressed (and I think it should be) I think it should be addressed in default registry, not the core spec.

aphillips added a commit to unicode-org/message-format-wg that referenced this pull request Apr 19, 2024
I found these errors while [commenting](unicode-org/icu#2973 (comment)) on an issue in ICU. Fixing the text for consistency.
@mihnita
Copy link
Contributor

mihnita commented Apr 19, 2024

In general the philosophy in how errors are handled in JS / Python and other other can be quite different from something like ICU.
If JS throws at runtime on some website, people complain, I fix my JS, and it's fixed in minutes.

If I do throw in ICU (C++ or Java) on something like a watch, it means a firmware update that has to be explicitly pushed to millions of devices.
So libraries like ICU have to be a lot more tolerant.

If the spec explicitly REQUIRES such mistakes to trigger errors, then I would strongly push to change the spec.

@catamorphism
Copy link
Contributor Author

@aphillips @mihnita When I say "the spec requires it", I mean in the sense that the tests are part of the spec.

I think I raised the question before of whether the test should be required to be an error, and Eemeli's comment here said yes, which is what I'm going off. But I'm happy to close the PR (for now) if there needs to be more discussion on the spec side.

@aphillips
Copy link
Member

@catamorphism Thanks for this.

@eemeli's comment and what the tests do might be the "right thing to do", but, as noted, they are not what the spec says. We need to be very careful about creating conformance tests. They need to match what the spec actually requires. I think it is valid for an implementation (ICU4C is one) to emit an error for this and render the fallback expression. But these are not syntax or data model errors.

aphillips added a commit to unicode-org/message-format-wg that referenced this pull request Apr 22, 2024
I found these errors while [commenting](unicode-org/icu#2973 (comment)) on an issue in ICU. Fixing the text for consistency.
@catamorphism catamorphism marked this pull request as draft April 22, 2024 18:23
@catamorphism
Copy link
Contributor Author

I'm converting this PR to draft since it's an ongoing topic of discussion in the spec working group.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants