Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comparator Errors in 4.32 I-Build: I20240324-1800 #1940

Closed
HannesWell opened this issue Mar 26, 2024 · 20 comments
Closed

Comparator Errors in 4.32 I-Build: I20240324-1800 #1940

HannesWell opened this issue Mar 26, 2024 · 20 comments
Labels
bug Something isn't working

Comments

@HannesWell
Copy link
Member

Since https://download.eclipse.org/eclipse/downloads/drops4/I20240324-1800/ there are two unanticipated comparator errors:

Comparator differences from current build
	/home/jenkins/agent/workspace/Builds/I-build-4.32/eclipse.platform.releng.aggregator/eclipse.platform.releng.aggregator/cje-production/siteDir/eclipse/downloads/drops4/I20240324-1800
compared to reference repo at 
	https://download.eclipse.org/eclipse/updates/4.32-I-builds

1.  eclipse.platform/team/bundles/org.eclipse.compare/.polyglot.META-INF
   no-classifier: different
      org/eclipse/compare/internal/Utilities.class: different
    The main artifact has been replaced with the baseline version.
    The following attached artifacts have been replaced with the baseline version: [sources]

2.  eclipse.pde/e4tools/bundles/org.eclipse.e4.tools.emf.ui/pom.xml
   no-classifier: different
      org/eclipse/e4/tools/emf/ui/common/ComboViewerAutoComplete$3.class: different
    The main artifact has been replaced with the baseline version.
    The following attached artifacts have been replaced with the baseline version: [sources]

While the first one is already resolved by a change in the affected bundle the second one still requires a enforced qualifier update to resolve. While it is easy to fix, I wonder if this is expected or not?

@HannesWell HannesWell added the bug Something isn't working label Mar 26, 2024
@HannesWell
Copy link
Member Author

I could not find a change in the code referenced in
https://github.com/eclipse-pde/eclipse.pde/blob/485750aa618ba615f2df9990f4ee974ac6a9eaad/e4tools/bundles/org.eclipse.e4.tools.emf.ui/src/org/eclipse/e4/tools/emf/ui/common/ComboViewerAutoComplete.java#L71_L101

that could cause the difference, so I wonder if this is due to a change in the Java-compiler?
In the meantime I'll enforce a qualifier update.

@sravanlakkimsetti
Copy link
Member

sravanlakkimsetti commented Mar 27, 2024

artifactcomparisons.zip

I see two problems.

  1. Generated attribute got reordered
    image

  2. There is a reorder in lambda parameters.
    image
    image

Not sure what caused these.
@mpalat @jarthana Need your input here

@MohananRahul
Copy link
Contributor

4.32 I-Build: I20240326-1800 - BUILD FAILED : seems infra issue , https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4475

@HannesWell
Copy link
Member Author

The original issue has been resolved.

I see two problems.

1. Generated attribute got reordered
2. There is a reorder in lambda parameters.

Not sure what caused these.
@mpalat @jarthana Need your input here

I think that can be handled by JDT if it is interesting, can't it?

@iloveeclipse
Copy link
Member

This seem to be a side effect of eclipse-jdt/eclipse.jdt.core#1181.

@srikanth-sankaran: could you please check if the synthetic method arguments order change was intended for synthetic methods generated for lambdas?

@srikanth-sankaran
Copy link

srikanth-sankaran commented Mar 29, 2024

This seem to be a side effect of eclipse-jdt/eclipse.jdt.core#1181.

@srikanth-sankaran: could you please check if the synthetic method arguments order change was intended for synthetic methods generated for lambdas?

I'll take a look, Thanks. @stephan-herrmann I may request for help if inference has a role. The change in the PR itself is to not shut the door, I can't see yet how that may result in a class file change.

@srikanth-sankaran
Copy link

The original issue has been resolved.

I see two problems.

1. Generated attribute got reordered
2. There is a reorder in lambda parameters.

Not sure what caused these.
@mpalat @jarthana Need your input here

I think that can be handled by JDT if it is interesting, can't it?

It is not actually attributes that got reordered - but field declarations.

@laeubi
Copy link
Contributor

laeubi commented Mar 30, 2024

It is not actually attributes that got reordered - but field declarations.

Could fields be ordered by name so they always have a consistent order regardless of position.

@srikanth-sankaran
Copy link

Analysis of differences in **ComboViewerAutoComplete$3** : Fields being reordered.

On master, first we discover the need for a synthetic field to hold the captured outer local dropDown in this call stack:

LocalTypeBinding(SourceTypeBinding).addSyntheticFieldForInnerclass(LocalVariableBinding) line: 270	
LocalTypeBinding(NestedTypeBinding).addSyntheticArgumentAndField(LocalVariableBinding) line: 118	
MethodScope(BlockScope).emulateOuterAccess(LocalVariableBinding) line: 393	
SingleNameReference.manageEnclosingInstanceAccessIfNecessary(BlockScope, FlowInfo) line: 930	
SingleNameReference.analyseCode(BlockScope, FlowContext, FlowInfo, boolean) line: 219	
MessageSend.analyseCode(BlockScope, FlowContext, FlowInfo) line: 157	
MessageSend(Expression).analyseCode(BlockScope, FlowContext, FlowInfo, boolean) line: 252	
MessageSend.analyseCode(BlockScope, FlowContext, FlowInfo) line: 157	
MessageSend(Expression).analyseCode(BlockScope, FlowContext, FlowInfo, boolean) line: 252	
MessageSend.analyseCode(BlockScope, FlowContext, FlowInfo) line: 157	
**LambdaExpression.analyzeExceptions() line: 570**	
LambdaExpression.cachedResolvedCopy(TypeBinding, boolean, boolean, InferenceContext18, Scope) line: 1012	
LambdaExpression.resolveExpressionExpecting(TypeBinding, Scope, InferenceContext18) line: 1030	
ConstraintExpressionFormula.reduce(InferenceContext18) line: 188	
BoundSet.reduceOneConstraint(InferenceContext18, ConstraintFormula) line: 975	
InferenceContext18.reduce() line: 1092	
InferenceContext18.solve(boolean, boolean) line: 1044	
InferenceContext18.solve(boolean) line: 1034	
ParameterizedGenericMethodBinding.computeCompatibleMethod18(MethodBinding, TypeBinding[], Scope, InvocationSite) line: 256	
ParameterizedGenericMethodBinding.computeCompatibleMethod(MethodBinding, TypeBinding[], Scope, InvocationSite) line: 92	
MethodScope(Scope).computeCompatibleMethod(MethodBinding, TypeBinding[], InvocationSite, boolean) line: 842	
MethodScope(Scope).computeCompatibleMethod(MethodBinding, TypeBinding[], InvocationSite) line: 799	
MethodScope(Scope).findMethod0(ReferenceBinding, char[], TypeBinding[], InvocationSite, boolean) line: 1753	
MethodScope(Scope).findMethod(ReferenceBinding, char[], TypeBinding[], InvocationSite, boolean) line: 1655	
MethodScope(Scope).getMethod(TypeBinding, char[], TypeBinding[], InvocationSite) line: 3145	
MessageSend.findMethodBinding(BlockScope) line: 1134	
MessageSend.resolveType(BlockScope) line: 910	
MessageSend(Expression).resolve(BlockScope) line: 1118	
MessageSend(Statement).resolveWithBindings(LocalVariableBinding[], BlockScope) line: 498	
ASTNode.resolveStatements(Statement[], BlockScope) line: 726	
MethodDeclaration(AbstractMethodDeclaration).resolveStatements() line: 715	
MethodDeclaration.resolveStatements() line: 410	
MethodDeclaration(AbstractMethodDeclaration).resolve(ClassScope) line: 613	
TypeDeclaration.resolve() line: 1514	
TypeDeclaration.resolve(BlockScope) line: 1618	
QualifiedAllocationExpression.getAnonymousConstructorBinding(ReferenceBinding, BlockScope) line: 665	
QualifiedAllocationExpression.resolveTypeForQualifiedAllocationExpression(BlockScope) line: 556	
QualifiedAllocationExpression.resolveType(BlockScope) line: 302	

And then discover the need for a synthetic field to hold the enclosing instance for ComboViewerAutoComplete in this call stack:

LocalTypeBinding(SourceTypeBinding).addSyntheticFieldForInnerclass(ReferenceBinding) line: 319	
LocalTypeBinding(NestedTypeBinding).addSyntheticArgumentAndField(ReferenceBinding) line: 131	
TypeDeclaration.manageEnclosingInstanceAccessIfNecessary(BlockScope, FlowInfo) line: 1054	
TypeDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 254	
QualifiedAllocationExpression.analyseCode(BlockScope, FlowContext, FlowInfo) line: 141	
LocalDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 109	
ConstructorDeclaration.analyseCode(ClassScope, InitializationFlowContext, FlowInfo, int) line: 183	
TypeDeclaration.internalAnalyseCode(FlowContext, FlowInfo) line: 953	
TypeDeclaration.analyseCode(CompilationUnitScope) line: 305	

But before eclipse-jdt/eclipse.jdt.core#1181 :
the need for synthetics are realized in this order: first the need for enclosing instance ComboViewerAutoComplete in this call stack

LocalTypeBinding(SourceTypeBinding).addSyntheticFieldForInnerclass(ReferenceBinding) line: 320	
LocalTypeBinding(NestedTypeBinding).addSyntheticArgumentAndField(ReferenceBinding) line: 131	
TypeDeclaration.manageEnclosingInstanceAccessIfNecessary(BlockScope, FlowInfo) line: 1051	
TypeDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 251	
QualifiedAllocationExpression.analyseCode(BlockScope, FlowContext, FlowInfo) line: 141	
LocalDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 109	
ConstructorDeclaration.analyseCode(ClassScope, InitializationFlowContext, FlowInfo, int) line: 183	
TypeDeclaration.internalAnalyseCode(FlowContext, FlowInfo) line: 950	
TypeDeclaration.analyseCode(CompilationUnitScope) line: 302	

AND THEN the need for a synthetic field to hold the captured outer local dropDown in this call stack:

LocalTypeBinding(SourceTypeBinding).addSyntheticFieldForInnerclass(LocalVariableBinding) line: 271	
LocalTypeBinding(NestedTypeBinding).addSyntheticArgumentAndField(LocalVariableBinding) line: 118	
MethodScope(BlockScope).emulateOuterAccess(LocalVariableBinding) line: 393	
SingleNameReference.manageEnclosingInstanceAccessIfNecessary(BlockScope, FlowInfo) line: 930	
SingleNameReference.analyseCode(BlockScope, FlowContext, FlowInfo, boolean) line: 219	
MessageSend.analyseCode(BlockScope, FlowContext, FlowInfo) line: 157	
MessageSend(Expression).analyseCode(BlockScope, FlowContext, FlowInfo, boolean) line: 252	
MessageSend.analyseCode(BlockScope, FlowContext, FlowInfo) line: 157	
LocalDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 109	
MethodDeclaration.analyseCode(ClassScope, FlowContext, FlowInfo) line: 172	
TypeDeclaration.internalAnalyseCode(FlowContext, FlowInfo) line: 958	
TypeDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 253	
QualifiedAllocationExpression.analyseCode(BlockScope, FlowContext, FlowInfo) line: 141	
LocalDeclaration.analyseCode(BlockScope, FlowContext, FlowInfo) line: 109	
ConstructorDeclaration.analyseCode(ClassScope, InitializationFlowContext, FlowInfo, int) line: 183	
TypeDeclaration.internalAnalyseCode(FlowContext, FlowInfo) line: 950	
TypeDeclaration.analyseCode(CompilationUnitScope) line: 302	

So, in the post eclipse-jdt/eclipse.jdt.core#1181 world, because LambdaExpression.analyzeExceptions() is NOT short circuited we discover the captured outer local first in Resolve phase itself and then the enclosing instance in Analyze phase.

In the pre eclipse-jdt/eclipse.jdt.core#1181 world, I could confirm that LambdaExpression.analyzeExceptions() is short circuited due to copy.hasIgnoredMandatoryErrors being true - so both are discovered in Analyze phase in the reversed order.

So this change is acceptable, explainable, expected and harmless.

Moving on to analyze the next diff in Utilities.class ... Stay tuned

@laeubi
Copy link
Contributor

laeubi commented Mar 30, 2024

So this change is acceptable, explainable, expected and harmless.

Can it maybe still considered to have a stable order independent of "discovery order"? e.g. sort all fields by their string name before emitting it to the class byte code?

@srikanth-sankaran
Copy link

Utilies class comparator error analysis: This is very similar to the above.

On master, we first discover the captured outer local event of type PropertyChangeEvent in this call stack. See the frame for LambdaExpression.analyzeExceptions() on the call stack.

LambdaExpression.addSyntheticArgument(LocalVariableBinding) line: 1317	
MethodScope(BlockScope).emulateOuterAccess(LocalVariableBinding) line: 374	
SingleNameReference.manageEnclosingInstanceAccessIfNecessary(BlockScope, FlowInfo) line: 930	
SingleNameReference.analyseCode(BlockScope, FlowContext, FlowInfo, boolean) line: 219	
SingleNameReference.analyseCode(BlockScope, FlowContext, FlowInfo) line: 188	
MessageSend.analyseCode(BlockScope, FlowContext, FlowInfo) line: 247	
LambdaExpression.analyzeExceptions() line: 570	
LambdaExpression.cachedResolvedCopy(TypeBinding, boolean, boolean, InferenceContext18, Scope) line: 1012	
LambdaExpression.resolveExpressionExpecting(TypeBinding, Scope, InferenceContext18) line: 1030	
ConstraintExpressionFormula.reduce(InferenceContext18) line: 188	
BoundSet.reduceOneConstraint(InferenceContext18, ConstraintFormula) line: 975	
InferenceContext18.reduce() line: 1092	
InferenceContext18.solve(boolean, boolean) line: 1044	
InferenceContext18.solve(boolean) line: 1034	
ParameterizedGenericMethodBinding.computeCompatibleMethod18(MethodBinding, TypeBinding[], Scope, InvocationSite) line: 256	
ParameterizedGenericMethodBinding.computeCompatibleMethod(MethodBinding, TypeBinding[], Scope, InvocationSite) line: 92	
BlockScope(Scope).computeCompatibleMethod(MethodBinding, TypeBinding[], InvocationSite, boolean) line: 842	
BlockScope(Scope).computeCompatibleMethod(MethodBinding, TypeBinding[], InvocationSite) line: 799	
BlockScope(Scope).findMethod0(ReferenceBinding, char[], TypeBinding[], InvocationSite, boolean) line: 1753	
BlockScope(Scope).findMethod(ReferenceBinding, char[], TypeBinding[], InvocationSite, boolean) line: 1655	
BlockScope(Scope).getMethod(TypeBinding, char[], TypeBinding[], InvocationSite) line: 3145	
MessageSend.findMethodBinding(BlockScope) line: 1134	
MessageSend.resolveType(BlockScope) line: 910	
MessageSend(Expression).resolve(BlockScope) line: 1118	
MessageSend(Statement).resolveWithBindings(LocalVariableBinding[], BlockScope) line: 498	
ASTNode.resolveStatements(Statement[], BlockScope) line: 726	
Block.resolve(BlockScope) line: 128	
ForeachStatement.resolve(BlockScope) line: 683	
ForeachStatement(Statement).resolveWithBindings(LocalVariableBinding[], BlockScope) line: 498	
ASTNode.resolveStatements(Statement[], BlockScope) line: 726	
Block.resolve(BlockScope) line: 128	
LambdaExpression.resolveType(BlockScope, boolean) line: 486	
LambdaExpression(FunctionalExpression).resolveType(BlockScope) line: 189	
LocalDeclaration.resolve(BlockScope, boolean) line: 402	
LocalDeclaration.resolve(BlockScope) line: 258	
LocalDeclaration(Statement).resolveWithBindings(LocalVariableBinding[], BlockScope) line: 498	
ASTNode.resolveStatements(Statement[], BlockScope) line: 726	
MethodDeclaration(AbstractMethodDeclaration).resolveStatements() line: 715	
MethodDeclaration.resolveStatements() line: 410	
MethodDeclaration(AbstractMethodDeclaration).resolve(ClassScope) line: 613	
TypeDeclaration.resolve() line: 1514	
TypeDeclaration.resolve(CompilationUnitScope) line: 1643	

and in the pre eclipse-jdt/eclipse.jdt.core#1181 world, I could confirm that LambdaExpression.analyzeExceptions() is short circuited due to copy.hasIgnoredMandatoryErrors being true - so both are discovered in Analyze phase in the reversed order.

So this change is acceptable, explainable, expected and harmless.

@srikanth-sankaran
Copy link

This seem to be a side effect of eclipse-jdt/eclipse.jdt.core#1181.

@srikanth-sankaran: could you please check if the synthetic method arguments order change was intended for synthetic methods generated for lambdas?

I can assure you the change was not intended as I don't want to cause or deal with comparator errors 😆

But the differences are acceptable, explainable, expected and harmless.

@srikanth-sankaran
Copy link

This seem to be a side effect of eclipse-jdt/eclipse.jdt.core#1181.
@srikanth-sankaran: could you please check if the synthetic method arguments order change was intended for synthetic methods generated for lambdas?

I can assure you the change was not intended as I don't want to cause or deal with comparator errors 😆

But the differences are acceptable, explainable, expected and harmless.

Basically we were short circuiting some code in the compiler viz LambdaExpression.analyzeExceptions() based on old and dated draft specification. The final version of spec withdrew some clauses but we didn't withdraw our implementation of the draft clauses. Now the short circuited code runs as it is not short circuited anymore and that results in some changes.

This batch of diffs is harmless.

@srikanth-sankaran
Copy link

srikanth-sankaran commented Mar 30, 2024

This batch of diffs is harmless.

While that is true, we are aware that by not short circuiting LambdaExpression.analyzeExceptions() (short circuiting - the pre eclipse-jdt/eclipse.jdt.core#1181 behavior - is the incorrect, spec-not-compliant behavior), we are running more code than we used to in the pre eclipse-jdt/eclipse.jdt.core#1181 world. Could that cause some unanticipated for side effects ? That is what eclipse-jdt/eclipse.jdt.core#2208 is for. Such a side effect showing up would NOT mean eclipse-jdt/eclipse.jdt.core#1181 is bad - It is after all removing behavior that was only compliant with draft JLS and NOT with final JLS. Any such side effects would have to be handled suitably when uncovered. That is all.

@srikanth-sankaran
Copy link

So this change is acceptable, explainable, expected and harmless.

Can it maybe still considered to have a stable order independent of "discovery order"? e.g. sort all fields by their string name before emitting it to the class byte code?

Not sure. Doing that is much more work upfront than having to deal with cases like this. I am also not sure what JVMS demands for declared fields. (Here we are dealing with synthetics.)

@srikanth-sankaran
Copy link

(BTW, this is a great example of a scenario where a compiler writer wouldn't think of looking into JDT build log themselves for a comparator errors - I was making a change far removed from code generation - or so I thought :)

So, there is value in having comparator errors reported one of the "checks" and have them exposed very visibly.)

@srikanth-sankaran
Copy link

It is not actually attributes that got reordered - but field declarations.

Could fields be ordered by name so they always have a consistent order regardless of position.

I don't think compilers can reorder fields in general - this will affect initialization sequence and exceptions raised etc. Here we are dealing with compiler inserted synthetics - so reordering is not an issue

@HannesWell
Copy link
Member Author

Thank you @srikanth-sankaran for checking.

(BTW, this is a great example of a scenario where a compiler writer wouldn't think of looking into JDT build log themselves for a comparator errors - I was making a change far removed from code generation - or so I thought :)

So, there is value in having comparator errors reported one of the "checks" and have them exposed very visibly.)

I think so as well. Of comparator-errors in the I-build can cause some tedious work when qualifiers need to be bumped, but it I don't think it is dramatic.
I mainly created the issue to raise awareness that something changed, not knowing if this was intended. :)

@stephan-herrmann
Copy link
Contributor

I agree that we don't have a problem here.

Still, we have the option to sort synthetic fields, since no spec defines their order. Had we done so in the past then eclipse-jdt/eclipse.jdt.core#1181 would not have created any bytecode difference.

@srikanth-sankaran , do we expect more such accidental changes of order, that would benefit from introducing ordering now? Or are we fine with the (small) risk of having to perform similar analysis again after future compiler changes cause similar bytecode differences?

@srikanth-sankaran
Copy link

I agree that we don't have a problem here.

Still, we have the option to sort synthetic fields, since no spec defines their order. Had we done so in the past then eclipse-jdt/eclipse.jdt.core#1181 would not have created any bytecode difference.

@srikanth-sankaran , do we expect more such accidental changes of order, that would benefit from introducing ordering now? Or are we fine with the (small) risk of having to perform similar analysis again after future compiler changes cause similar bytecode differences?

Even with synthetics we don't have complete control over order. For example anonymous class constructors must emit enclosing instances ahead of captured locals.

This is because the code invoking the constructor may be emitted by a different compiler than the one that emitted the constructor body.

Such a case doesn't arise with lambdas. So this may end up being a special case where we have a little bit freedom.

I will leave the sleeping dog lie 😊

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

7 participants