Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distinguish between type syntax vs expression syntax in the parser #13671

Merged
merged 2 commits into from Mar 27, 2024

Conversation

jeskew
Copy link
Contributor

@jeskew jeskew commented Mar 21, 2024

Resolves #13618

In order to make type determinations, the DeclaredTypeManager needs to know whether it is within type syntax or expression syntax. This distinction is meaningful for property access (e.g., <array type>[*] is legal within type syntax but not within expression syntax, whereas <array value>[?0] is legal within expression syntax but not within type syntax), variable access (variables within type syntax can refer to other types but not values, and variables within expression syntax can refer to values but not types), nullability control syntax, and literal value syntax.

Prior to Bicep 0.26, the type manager was relying on internal state that assumed the AST would be visited in order. This resulted in unstable type assignments if the type of type syntax was requested out of order. In Bicep 0.26.54, this was replaced with a deterministic check that used the model's syntax hierarchy to determine whether a given syntax node was a descendant of the type declaration of an output or parameter statement or the assigned value of a type statement (indicating type syntax) or not (indicating expression syntax). However, querying the syntax hierarchy can throw an exception if a node was not part of the original model (e.g., if it was created by a SyntaxRewriteVisitor).

This PR moves the responsibility for making this distinction to the parser. The parser is already tracking whether it is within type syntax or expression syntax, as the grammar for each syntax class is distinct. This PR introduces some new descendent classes of TypeSyntax that are minimally changed copies of descendent classes of ExpressionSyntax: VariableAccessSyntax is a descendent of ExpressionSyntax, and TypeVariableAccessSyntax is a descendent of TypeSyntax, but the two classes are otherwise identical. This means that the compiler can determine whether an arbitrary syntax node was encountered in type syntax or expression syntax based solely on the C# type of the syntax node rather than having to rely on the syntax hierarchy or order of operations.

About 50% of the line count for this PR comes from baseline changes. These are all to the main.syntax.bicep artifacts and consist of the class name for specific nodes being changed (e.g., VariableAccessSyntax -> TypeVariableAccessSyntax, StringSyntax -> StringTypeLiteralSyntax, etc.).

Microsoft Reviewers: Open in CodeFlow

Copy link
Contributor

github-actions bot commented Mar 21, 2024

Test this change out locally with the following install scripts (Action run 8458462785)

VSCode
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 8458462785
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 8458462785"
Azure CLI
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 8458462785
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 8458462785"

Copy link
Contributor

Test Results

    66 files   -     33      66 suites   - 33   22m 37s ⏱️ - 22m 31s
10 719 tests  -     20  10 717 ✅  -     20  2 💤 ±0  0 ❌ ±0 
25 334 runs   - 12 663  25 330 ✅  - 12 661  4 💤  - 2  0 ❌ ±0 

Results for commit d9083ef. ± Comparison against base commit 6f3a864.

@@ -1037,13 +1039,13 @@ protected IEnumerable<Token> NewLines()
}
}

private IntegerLiteralSyntax NumericLiteral()
private (Token literal, ulong value) NumericLiteral()
Copy link
Collaborator

@majastrz majastrz Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

private

Do we also need to make updates to the grammar.md pseudo grammar file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The grammar isn't changing, just the AST. So where previously we would have always used IntegerLiteralSyntax, we now use IntegerLiteralSyntax in a value expression and IntegerTypeLiteralSyntax in a type expression. That way there's no ambiguity later in the compilation process as to whether this syntax node was part of a type or a value.

Copy link
Collaborator

@majastrz majastrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link
Contributor

@shenglol shenglol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@jeskew jeskew merged commit 6b0e259 into main Mar 27, 2024
44 checks passed
@jeskew jeskew deleted the jeskew/13618 branch March 27, 2024 20:50
StephenWeatherford pushed a commit that referenced this pull request Apr 1, 2024
…13671)

Resolves #13618

In order to make type determinations, the `DeclaredTypeManager` needs to
know whether it is within type syntax or expression syntax. This
distinction is meaningful for property access (e.g., `<array type>[*]`
is legal within type syntax but not within expression syntax, whereas
`<array value>[?0]` is legal within expression syntax but not within
type syntax), variable access (variables within type syntax can refer to
other types but not values, and variables within expression syntax can
refer to values but not types), nullability control syntax, and literal
value syntax.

Prior to Bicep 0.26, the type manager was relying on internal state that
assumed the AST would be visited in order. This resulted in unstable
type assignments if the type of type syntax was requested out of order.
In Bicep 0.26.54, this was replaced with a deterministic check that used
the model's syntax hierarchy to determine whether a given syntax node
was a descendant of the type declaration of an `output` or `parameter`
statement or the assigned value of a `type` statement (indicating type
syntax) or not (indicating expression syntax). However, querying the
syntax hierarchy can throw an exception if a node was not part of the
original model (e.g., if it was created by a `SyntaxRewriteVisitor`).

This PR moves the responsibility for making this distinction to the
parser. The parser is already tracking whether it is within type syntax
or expression syntax, as the grammar for each syntax class is distinct.
This PR introduces some new descendent classes of `TypeSyntax` that are
minimally changed copies of descendent classes of `ExpressionSyntax`:
`VariableAccessSyntax` is a descendent of `ExpressionSyntax`, and
`TypeVariableAccessSyntax` is a descendent of `TypeSyntax`, but the two
classes are otherwise identical. This means that the compiler can
determine whether an arbitrary syntax node was encountered in type
syntax or expression syntax based solely on the C# type of the syntax
node rather than having to rely on the syntax hierarchy or order of
operations.

About 50% of the line count for this PR comes from baseline changes.
These are all to the `main.syntax.bicep` artifacts and consist of the
class name for specific nodes being changed (e.g.,
`VariableAccessSyntax` -> `TypeVariableAccessSyntax`, `StringSyntax` ->
`StringTypeLiteralSyntax`, etc.).

###### Microsoft Reviewers: [Open in
CodeFlow](https://microsoft.github.io/open-pr/?codeflow=https://github.com/Azure/bicep/pull/13671)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants