Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Upgrade Pitest to 1.15.8 #224

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

[WIP] Upgrade Pitest to 1.15.8 #224

wants to merge 11 commits into from

Conversation

echebbi
Copy link
Collaborator

@echebbi echebbi commented Mar 26, 2024

Closes #221 by upgrading Pitest to 1.15.8

This upgrade comes with several API breaking changes. A notable change is the removal of several "research-oriented" mutators from PIT and which are now wrapped in a dedicated PIT plug-in. See hcoles/pitest#993 for the changes and #221 (comment) for our decision to discard those mutators for now.

@echebbi echebbi added an: enhancement 📈 Improve something for: pitest 🐦 Related to PIT integration is: in progress 🚧 Someone is working on it labels Mar 26, 2024
@echebbi echebbi self-assigned this Mar 26, 2024
@echebbi echebbi linked an issue Mar 26, 2024 that may be closed by this pull request
@LorenzoBettini
Copy link
Collaborator

@echebbi I created #226 and build is green. Please have a look and merge it if you're OK with that.

I tried to cherry pick some of your commits but of you course you should expect merging conflicts.

I wonder if some of the failures I see in your builds (unbound Java 1.8 classpath container) might be due to Java 21 not supporting Java 8 anymore. In any case, I think the main build should be with Java 17 and we could have another build testing that, at least the runtime tests are green.

Copy link
Collaborator

@LorenzoBettini LorenzoBettini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@echebbi Thanks a lot for this great effort of porting to the new API! :)

I left a few comments, questions and required changes.

You may also want to first merge #228 and then rebase this one.

@@ -162,94 +144,31 @@ public void checkAllMutants() { // NOSONAR
PAGES.getRunMenu().checkAllMutators(TEST_CONFIG_NAME);
// run test and confirm result is as expected
PAGES.getRunMenu().runPitWithConfiguration(TEST_CONFIG_NAME);
coverageReportGenerated(TESTED_CLASSES, COVERAGE, 1, 84, 1);
coverageReportGenerated(TESTED_CLASSES, COVERAGE, 5, 20, 1);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We lose all these mutators because of the mutators removed by default in PIT?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like it, yes. We did lose a lot of mutations operators (see Mutations.java, MutatorApiOfPitTest.java and the list of research mutators). That being said I believe we should take some time to make sure I didn't miss something while upgrading.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@echebbi So creating a new bundle for https://github.com/pitest/pitest-rv-plugin and adding it to our distribution would bring the old mutators back?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just like we did for the pitest-junit5 plugin. Do you think that we should integrate the pitest-rv plugin after all?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@echebbi maybe we should/could. I'm a bit worried by the fact that people using Pitclipse would have more mutators than those using Maven and ignoring the new pitest-rv, but maybe we can rely on them knowing the Maven plugin as well.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default we run Pitest's DEFAULT mutator group so our mutators set would still be the same as a bare Pitest configuration. pitest-rv's mutators would only be exposed through the preferences/configuration pages which feels fair to me.

We can even add a column to the mutators table that explicitely states the origin of each mutator:

Name Description Contributor
Conditionals Boundary Replaces the relational operators <, <=, >, >= pitest
Negation Replaces any use of a numeric variable (local variable, field, array cell) with its negation pitest-rv-plugin

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@echebbi yes that'd be useful; but would such information be easily (automatically) retrieved and easy to maintain?

Moreover, with pitest-rv we would restore the "old defaults" or not?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would such information be easily (automatically) retrieved and easy to maintain?

It might be possible to retrieve/infer the information through reflection. My intuition is that since we have access to the MethodFactoryFactory instances (each mutation operator is implemented as a child of MethodMutatorFactory) we should be able to get the info with something along the lines of:

// maps plugin ID to the name we want to display in the table
//
Map<String, String> pluginIdToContributorName = Map.of(
    "org.pitest", "pitest",
    "org.pitest.pitest-rv-plugin", "pitest-rv"
);
Collection<MethodMutatorFactory> mutators = ServiceLoader.load(MethodMutatorFactory.class, IsolationUtils.getContextClassLoader());

for (MethodMutatorFactory mutator : mutators) {
    //
    // retrieve the contributor through reflection
    //
    Bundle bundle = FrameworkUtil.getBundle(mutator.getClass());
    //
    // update our 'Mutators' enum with the contributor name
    //
    String contributorName = pluginIdToContributorName.get(bundle.getBundleId());
    Mutators.valueOf(mutator.getName()).setContributorName(contributorName);
}

Or, if we want to be really extensible it should be possible to craft a solution based on a custom extension point so that external plug-ins can provide new mutators without having to change a single piece of code in Pitclipse's core. That would require some refactoring though so I believe that such a work would deserve its own PR.

Moreover, with pitest-rv we would restore the "old defaults" or not?

We wouldn't because this group is not related to the research operators and has been completely removed. We may want to somehow support the groups related to research operators though.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@echebbi thanks for the clarification, especially about the old defaults, which are now gone for good. I thought the pitest-rv was related to that so I was reluctant to add pitest-rv right away because that would have required to undo the removal of old defaults, but it's not like that :)

So, shall we address pitest-rv in a new PR and go on with this one?

I would also try to build with Java 21 and have a UI test with Java 21, but we can do that in another PR as well.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we address pitest-rv in a new PR and go on with this one?

@LorenzoBettini fine by me!

@echebbi
Copy link
Collaborator Author

echebbi commented Apr 1, 2024

Finally a green build 🎉@LorenzoBettini thanks for your help :) I'm not 100% confident regarding the new mutators/groups set and the current results though. I think we should take some time to make sure everything is ok before merging.

Copy link

sonarcloud bot commented Apr 2, 2024

@LorenzoBettini
Copy link
Collaborator

@echebbi shall we merge this and release?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
an: enhancement 📈 Improve something for: pitest 🐦 Related to PIT integration is: in progress 🚧 Someone is working on it
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Runing pitclipse with Eclipse 2023-12 and Java 21 fails
2 participants