"Skipped parameter" warnings in jenkins server log #2774
Comments
|
FYI @andrew-m-leonard @smlambert (Shelley I sent you a log from this a while back and I think there may be fewer such messages now but unclear if that's my memory or if it was a slightly different error! |
|
Also tagging @sophia-guo since she was involved in the discussions on the community call earlier today :-) |
|
According to the discussion if jenkins jobs are triggered with undefined parameters there will be those Initially when new features was added there was a period that aqa-tests or smoke tests jobs were triggered with undefined parameters. For example, USE_TESTENV_PROPERTIES, the reason is testJobTemplate was not updated accordingly as it didn't affect the functionalities. I have double checked all parameters of aqa-tests and smoke tests, all of them have been updated in testJobTemplate. That means if there is still those warning message related with aqa-tests or smoketests regenerating those jobs should fix it. Note autoGen is enabled with aqa-tests and disabled with smoketests. |
|
Thanks for checking @sophia-guo! It looks like the remaining ones are mostly on the build pipelines so the test jobs may now be all sorted. I've just done a check on the recent logs and these are the remaining warnings - interestingly only the (new) jdk19u pipelines are showing up here @andrew-m-leonard: |
|
@sxa I don't understand the above, none of the build jobs have a Parameter called NODE_LABEL, they only have 4 paramaters:
|
|
That sounds consistent with the warning message - "it is undefined on " but presumably something is trying to start them using those extra, unusable, parameters. |
|
Had a look at the log for something else today. We've still got:
|
|
Regenerated the following test jobs: JDK 11,17,19 hotspot/platforms JDK 8 hotspot/platforms |
|
@smlambert Most of these are looking good now, but Test_openjdk8_dragonwell_sanity.system_aarch64_linux is still throwing messages about |
|
A number of the Similarly for create_installer_mac it is hard coded to @andrew-m-leonard I suspect there must be other places in there where we're passing in a |
|
@andrew-m-leonard After merging adoptium/ci-jenkins-pipelines#668 we've still got a few remaining occurrences on the PR tester jobs which will need addressing: Plus these in other build jobs: NOTE: Since was the log of the last day the presence of mostly jdk17 jobs is likely because of when that versions was scheduled and should not be assumed to mean that only that versions is affected |
|
@smlambert @sophia-guo This is the remaining list from the test jobs. Interestingly most of these seem to be on JDK17 jobs. A number of the Here's the full list from the last day's log: NOTE: Since was the log of the last day the presence of mostly jdk17 jobs is likely because of when that versions was scheduled and should not be assumed to mean that only that versions is affected |
Can someone confirm the above to me please? If there is an easy way to trigger this that doesn't involve running a full openjdkXX-pipeline with the aqaAutoGen flag? Can I just run https://ci.adoptium.net/job/Test_Job_Auto_Gen/build?delay=0sec with the appropriate flags for the other variants to clear them up? This seems like an easy fix if it's all that's required - are there any potential side-effects from doing that? |
|
I've kicked off these two runs: This will cover most of the ones we have remaining from a test perspective today, although a few others are notable in the logs too ... @smlambert @sophia-guo are these still needed and if not can you delete the jobs from the jenkins server please?
|
|
I don't have any permission to delete jenkin jobs. |
|
We also have the problem of TRSS getting confused when we delete Jenkins jobs, so we should programatically regenerate them without deletion or else we end up with this type of problems, adoptium/aqa-test-tools#860 |
Which ohes would you like deleting? I can action those if required (Shelley can too!)
I believe the problem in there was caused by deletion of jobs that were still in use which then restarted from a build number of 1. In these cases I'm asking if the jobs are still required, or if they can be removed without replacement, which I do not believe is related to the problem in 860, but if I'm incorrect please let me know. If there is a specific process for the regeneration other than what I've done today in the above comment please let me know :-) The question here is whether these jobs are required at all i.e. whether they can be removed from jenkins and TRSS (I'm unclear if any of these ones are monitored by TRSS). |
|
Getting on top of these now I think - there were over 5000 yesterday, but as of half way through today we've only had 8. I'll do another summary over the weekend. We could do with regenerating the @andrew-m-leonard is there a way to point the regen jobs at a configuration that includes the non-temurin ones? There have also been instances of these on the pr-tester jobs which could probably do with a bit of maintenance (although they're not showing up in the log today so far) |
|
So I think the other variants need to be in the pipelines targetConfiguration to be re-gen'd so If you could point the regen at a fork with such a change you could do possibly? |
|
Yeah that's what I wanted to do but it wasn't clear how to make that work. Is there a way to point it at an alternate location or would I have to create a new branch, put the changes in, edit the generator job (always a risk!) to use the fork, and then run it ... Or is there a simpler way? There is DEFAULTS_URL which can point externally (although the default in the description of that points to something at the old AdoptOpenJDK repo, but I don't think the equavalint file in the new repo is the one I want) |
|
From today and yesterday (Note this may not be an exhaustive list since it's not necessarily the same each day but gives an idea. We're getting close now though!): Build jobs (11)Test jobs (28) J9(16)/Dragonwell(9)/Bisheng(3)Test jobs (21) - HotSpot/Grinder |
|
The test autogen jobs seem to have two parameters causing problems too:
|
|
@smlambert @andrew-m-leonard Is there a way we can regen the jobs from the earlier comment (Plus the pr-tester ones which can show up problems)> |
A fair comment - I hadn't actually spotted that it was on non-current ones. If you confirm you're ok with us just deleting them I'm happy to go through and do that as a background task. That would likely just leave things like this on the PR tester jobs: |
From a comment from Shelley in a meeting earlier today it seems likely that running https://ci.eclipse.org/temurin-compliance/job/AQA_Test_Pipeline with a (Although I'm not sure if this will regen the Similarly I'm unclear if we have a way to regen the various Grinder variants (although almost all of those in the twisty earlier were |
|
Runs - noting that the AQA_Test_pipeline job does not have an option for
This will cover most of the outstanding test jobs other than Grinders, |
|
Noting also that we're getting a couple of the parameters giving warnings on
(I've had 3100 of those lines from Test_Job_Auto_Gen in the last two hours since I kicked off those jobs in the previous comment) |
|
FYI, TEST_JOB_NAME only applies to AQA_Test_Pipeline, there is no parameter of that name used in the generated jobs (nor do we want their to be). |
I have been slowly removing grinder variants especially if they were last run 1 yr or more ago |
Makes sense, although the warning would suggest that |
The code cycles through all parameters and passes them down (with some exceptions at https://github.com/adoptium/aqa-tests/blob/master/buildenv/jenkins/aqaTestPipeline.groovy#L110-L114). It can be added to the set of exceptions that are not added to |
|
Deep dive into one of them (Corretto JDK8 sanity/system) for reference. There are only 13 entries relating to test skipped parameters today so far, and 8 of them are from different parameters on this job so yesterday's regens appear to have made quite a big difference.
|
@smlambert Can "suffixed" jobs like https://ci.adoptium.net/job/Test_openjdk11_hs_sanity.external_s390x_linux_system-test/ be removed too? The ones I've looked at today seem to be ones that haven't been run in nearly 3 years so I guess they were likely generated as some sort of experiment and are no longer required. I don't want to be deleting them myself though - would rather someone from the test side with knowledge of the particular jobs that should be in scope was able to handle it. |
|
I have a script that will look for jobs that have not been run in XX number of months, and optionally delete them if 'deleteJobs' parameter equals true. I think we should run such a job occasionally on the server to cull old, not-used jobs. I will do a pass in the coming weeks. |
These aren't new but were discovered while working on #2108
There are a lot of warnings in the jenkins log which we should look at clearing up in the interests of avoid the risk of being unable to see the wood for the trees. All are related to
NODE_LABEL's use in various places.Presumably the parameters are being passed in from the upstream job somewhere but they are not defined on the callee so are superfluous. Can we wasily stop 'rogue' parameters being passed in or should we set the
-Dmentioned in the error?90 WARNING hudson.model.ParametersAction#filter: Skipped parameter
NODE_LABELas it is undefined onbuild-scripts/release/create_installer_mac. Set-Dhudson.model.ParametersAction.keepUndefinedParameters=trueto allow undefined parameters to be injected as environment variables or-Dhudson.model.ParametersAction.safeParameters=[comma-separated list]to whitelist specific parameter names, even though it represents a security breach or-Dhudson.model.ParametersAction.keepUndefinedParameters=falseto no longer show this message.The text was updated successfully, but these errors were encountered: