plugin incorrectly filters out project's jar and test-jar

[shawnmckinney]

Problem:

Jmeter plugin will filter out testPlan libraries when setting both the project's main and test jars. For example:

<testPlanLibraries>
  <artifact>org.apache.directory.fortress:fortress-core:jar:tests:${project.version}</artifact>
  <artifact>org.apache.directory.fortress:fortress-core:${project.version}</artifact>
  ...
</testPlanLibraries>

When I run a test, will get:

2022-01-14 15:54:31,095 ERROR o.a.j.JMeter: Uncaught exception in thread Thread[StandardJMeterEngine,5,main]
java.lang.NoClassDefFoundError: org/apache/directory/fortress/core/SecurityException

Note, that class is contained within the fortress-core artifact. So when I view the contents of

$ ls target/9998d2e4-c17b-4992-b3c5-c925e4651ae9/jmeter/lib/fortress-core*
target/9998d2e4-c17b-4992-b3c5-c925e4651ae9/jmeter/lib/fortress-core-2.0.8-SNAPSHOT-tests.jar

Older version of the plugin didn't have this problem:

<plugin>
  <groupId>com.lazerycode.jmeter</groupId>
  <artifactId>jmeter-maven-plugin</artifactId>
  <version>1.10.1</version>
  ...
            <dependencies>
              <dependency>
                <groupId>org.apache.directory.fortress</groupId>
                <artifactId>fortress-core</artifactId>
                <version>${project.version}</version>
              </dependency>
              <dependency>
                <groupId>org.apache.directory.fortress</groupId>
                <artifactId>fortress-core</artifactId>
                <type>test-jar</type>
                <version>${project.version}</version>
              </dependency>

$ ls target/jmeter/lib/fortress-core*
target/jmeter/lib/fortress-core-2.0.8-SNAPSHOT.jar  target/jmeter/lib/fortress-core-2.0.8-SNAPSHOT-tests.jar

The older version is the correct behavior. There is no workaround for this problem that I know of. It prevents projects from pulling from both the main and tests jars for running jmeter.

I also described this problem on the ML but have not yet received a response:
https://groups.google.com/g/maven-jmeter-plugin-users/c/HhH7T1Q34M8

I'm happy to help test and document a fix, as needed, but don't have any experience developing maven plugins. Would be willing to take this on if someone from the project would mentor.

Thanks

--
Shawn

[Ardesco]

Can you let me know which version of the plugin you are using? 3.5.0?

[shawnmckinney]

Yes, 3.5.0. I'm doing some testing with latest as we speak. Stepping thru the plugin code w/ the debugger. It looks like I misspoke. The project's main jar isn't being removed as a duplicate. It isn't being added at all. Still trying to piece together why.

I've a couple of projects to test in, one has the setup described earlier, with the two project dependencies, one of which is test-jar.

The second project doesn't have a test jar. All of its dependencies are pulled into target/.../jmeter/lib. But again, the jar that's produc
ed by the pom &/or its classes aren't.

Here's a link to project's pom.xml.

Sorry if this is unclear, still working through it myself. Thanks for replying.

--
Shawn

[Ardesco]

So i'm slightly confused after looking at your POM's, I don't see anything relating to test JARs.

Are you trying to build a project, and then run the jmeter-maven-plugin all in the same pom?

If this is the case, I suspect what is happening is that the configure part of the jmeter-maven-plugin execution happens in the compile phase. At this point if you are trying to build a local project in the same pom your code will be getting compiled at the same time as JMeter is configured. This means that any JARs that are created by you project in the compile phase will not yet be available to the jmeter-maven-plugin as they will not be installed into your local maven repository until the install phase has been completed.

(Link to the list of phases to help visualise this - https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html).

I would expect the POM that configures and runs the jmeter-maven-plugin to be a sub-project that depends upon your main project being built, or just an entirely separate project (with the caveat that you would need to build the other project locally first to ensure that your test jar has been installed into your local maven repository).

As for the artifact definition, I think you just need this.

<testPlanLibraries>
    <artifact>org.apache.directory.fortress:fortress-core:tests:${project.version}</artifact>
    <artifact>org.apache.directory.fortress:fortress-core:${project.version}</artifact>
</testPlanLibraries>

[shawnmckinney]

So i'm slightly confused after looking at your POM's, I don't see anything relating to test JARs.

There are two projects that I'm testing the jmeter maven plugin:

1. Apache Fortress Core - this project's pom creates two jars, one test, the other main.
2. Ldap Load Gen - this project's pom creates just one jar. No test jar.

So, as I'm working thru the problem I first reported on #1, on the mailing list and here. Later, in order to rule out any sort of conflict between the two jars, I switched to #2, which was linked to earlier on this ticket.

Are you trying to build a project, and then run the jmeter-maven-plugin all in the same pom?

Yes, that is exactly what I'm trying to do. I mentioned earlier, this 'used' to work, with an earlier plugin, 1.10.1 is the last known version where this behavior was possible.

If this is the case, I suspect what is happening is that the configure part of the jmeter-maven-plugin execution happens in the >compile phase. At this point if you are trying to build a local project in the same pom your code will be getting compiled at the >same time as JMeter is configured. This means that any JARs that are created by you project in the compile phase will not yet >be available to the jmeter-maven-plugin as they will not be installed into your local maven repository until the install phase has >been completed.

You're right of course and I'm embarrassed for overlooking this as I've been working my way thru this issue. Initially, on the older version of the plugin, I added a maven profile, that (I'll need to verify this) executed after the install phase, and so it all worked, for both test case 1 and 2 (from above).

I would expect the POM that configures and runs the jmeter-maven-plugin to be a sub-project that depends upon your main >project being built, or just an entirely separate project (with the caveat that you would need to build the other project locally first >to ensure that your test jar has been installed into your local maven repository).

I'm trying to avoid this if possible but I suppose that's a fallback option.

As for the artifact definition, I think you just need this.

org.apache.directory.fortress:fortress-core:tests:${project.version}
org.apache.directory.fortress:fortress-core:${project.version}

Yes, this is precisely how I'm configuring the dependencies for test case #1, as I pointed out both in ticket and on the ML. Again, #2 doesn't have a test jar. What's further confusing this discussion is the 2nd project, ldap load gen, depends on the first, i.e. uses the fortress-core.jar. So, its test plan dependencies are:

com.symas:ldap-load-gen:${project.version}
org.apache.commons:commons-pool2:${commons-pool2.version}

fortress-core isn't added here because its listed as a dependency of the project, further up in the pom.

Thanks again for having a look. I definitely appreciate the guidance here.

--
Shawn

[Ardesco]

The older version of the plugin (1.x.x) didn't use mavens phases correctly. This was changed with the version 2.0.0 release of the plugin.

In that old version it would do everything in the integration-test phase (Download jars, configure JMeter and then run tests). This meant that if you were trying to run multiple different tests it would clobber your output, and if you were trying to run different tests with different JMeter configurations in parallel the second test would reconfigure JMeter in the same directory while the first test was running (causing various issues). It also means that rerunning a test would always download and reconfigure JMeter (which isn't really needed).

The plugin now configures the various JMeter setups that you have specified in the compile phase, stores a link to that configuration in the config.json in your target directory and then runs the tests in the integration-test phase.

You may be able to work around the issue by doing the following (note the change in the configuration execution ID, it has been moved from configure to pre-integration-test, the configure name is mapped to the compile phase - (https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/blob/main/src/main/java/com/lazerycode/jmeter/mojo/ConfigureJMeterMojo.java#L49)[https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/blob/main/src/main/java/com/lazerycode/jmeter/mojo/ConfigureJMeterMojo.java#L49]):

<plugin>
    <groupId>com.lazerycode.jmeter</groupId>
    <artifactId>jmeter-maven-plugin</artifactId>
    <version>3.5.0</version>
    <executions>
        <!-- Generate JMeter configuration -->
        <execution>
            <id>configuration</id>
            <goals>
                <goal>pre-integration-test</goal>
            </goals>
        </execution>
        <!-- Run JMeter tests -->
        <execution>
            <id>jmeter-tests</id>
            <goals>
                <goal>jmeter</goal>
            </goals>
        </execution>
        <!-- Fail build on errors in test -->
        <execution>
            <id>jmeter-check-results</id>
            <goals>
                <goal>results</goal>
            </goals>
        </execution>
    </executions>
</plugin>

[shawnmckinney]

Links to both project's using jmeter-maven-plugin v1.10.1:

1. Apache Fortress Core
2. Ldap Load Gen

Here are profiles to execute the test cases.

[Ardesco]

Not sure what you want me to do with this, have you tried the suggestion above?

[shawnmckinney]

The plugin now configures the various JMeter setups that you have specified in the compile phase, stores a link to that >configuration in the config.json in your target directory and then runs the tests in the integration-test phase.

Makes total sense now!

You may be able to work around the issue by doing the following (note the change in the configuration execution ID, it has been >moved from configure to pre-integration-test, the configure name is mapped to the compile phase

Made this change:

<executions>
<execution>
  <id>configuration</id>
  <goals>
    <goal>pre-integration-test</goal>
  </goals>
</execution>
...

Getting this error:

[ERROR] Could not find goal 'pre-integration-test' in plugin com.lazerycode.jmeter:jmeter-maven-plugin:3.5.0 among available goals configure, gui, help, jmeter, remote-server, results -> [Help 1]

Am I missing something?

Not sure what you want me to do with this, have you tried the suggestion above?

I posted that before your latest reply. Just wanted to give links to earlier test cases where it worked. Disregard, as I know understand why it worked before.

--
Shawn

[shawnmckinney]

Probably should state that my maven skills are not good. Having said that, for this to work:

<execution>
  <id>configuration</id>
  <goals>
    <goal>pre-integration-test</goal>
  </goals>

I would expect a corresponding 'task' in the plugin code. As per the link you sent:

@Mojo(name = "configure", defaultPhase = LifecyclePhase.COMPILE)
public class ConfigureJMeterMojo extends AbstractJMeterMojo {

Maven can't find a task with a name pre-integration-test hence the error from the runtime.

But, I'm really out of my depth here. Currently RTFM. This one has me stumped and not going to give up until I have full understanding. Maybe I can help other neophytes with a blog post when I'm done.

Building a list of things that have stumped me. For example, the way the depenencies are listed, howto pulling in test-jar, other little tidbits that are probably obvious to the experienced but confusing to the newbie.

--
Shawn

[shawnmckinney]

Making changes to plugin:

@Mojo(name = "pre-integration-test", defaultPhase = LifecyclePhase.INTEGRATION_TEST)
public class ConfigureJMeterMojo extends AbstractJMeterMojo {

The projects jar(s) now included from testPlanLibs under target/jmeter/.../libs

Obviously, this isn't too good of a workaround right? We certainly can't break backward compatibility by replacing these tags. Is there a viable longterm solution here w/out me having to maintain a fork? How can we alter the plugin to support this w/out violating compatibility or best practices?

Still noodling around in the code. Did I say this is fun? :-)

[Ardesco]

Sorry my fault, I should have checked before pasting the last config instead of going from my obviously faulty memory. It's not the goal you need to change, you actually need to add a phase to the execution:

<plugin>
    <groupId>com.lazerycode.jmeter</groupId>
    <artifactId>jmeter-maven-plugin</artifactId>
    <version>3.5.0</version>
    <executions>
        <!-- Generate JMeter configuration -->
        <execution>
            <id>configuration</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>configure</goal>
            </goals>
        </execution>
        <!-- Run JMeter tests -->
        <execution>
            <id>jmeter-tests</id>
            <goals>
                <goal>jmeter</goal>
            </goals>
        </execution>
        <!-- Fail build on errors in test -->
        <execution>
            <id>jmeter-check-results</id>
            <goals>
                <goal>results</goal>
            </goals>
        </execution>
    </executions>
</plugin>

[shawnmckinney]

Sorry my fault, I should have checked before pasting the last config instead of going from my obviously faulty memory.

No worries. I'm learning, so it's all good.

It's not the goal you need to change, you actually need to add a phase to the execution

This worked for me. Have a fair amount of testing to do but I think you've got me on the right path. Thanks for taking the time it has been beneficial for me and now these projects are on the latest jmeter plugin which is a very good thing.

I suppose this issue can be closed now. It was my understanding that was faulty, not the plugin.

--
Shawn

[Ardesco]

I should probably document this for others first before closing this off.

[shawnmckinney]

Agreed. Another point of confusion is how to include the test jars. This is what worked for me:

<artifact>org.apache.directory.fortress:fortress-core:jar:tests:${project.version}</artifact>

I'm willing to help if need be.