It is impossible to mock exisiting local function in declarative pipeline

[grzegorzgrzegorz]

Hi,
Given is the pipeline which contains function inside the simpleDeclarativePipeline.pipeline file:

pipeline {
    agent { node { label 'test1' } }
    stages {
      stage('sdfsdf') {
        steps {
          githubNotify status: 'PENDING', context: 'Pipeline'
            runFunc()
        }
      }
    }
  }

  def runFunc(){
      echo "inside runFunc"
      doSomeStuff()
  }

When runFunc is mocked:

void setUp() throws Exception {

        baseScriptRoot = 'unit-testing-jenkinsfile'
        scriptRoots += 'src/main/groovy'
        scriptExtension = 'pipeline'

        super.setUp()

        helper.registerAllowedMethod("runFunc")
        helper.registerAllowedMethod("githubNotify", [Map.class], null)

    }

    @Test
    void should_execute_without_errors() throws Exception {
        def script = runScript("simpleDeclarativePipeline.pipeline")
        printCallStack()
        assertJobStatusSuccess()
    }

Then mocked function is not executed but real one instead and thus exception is thrown:
groovy.lang.MissingMethodException: No signature of method: simpleDeclarativePipeline.doSomeStuff() is applicable for argument types: () values: []

When runFunc inside pipeline is deleted:

 pipeline {
    agent { node { label 'test1' } }
    stages {
      stage('sdfsdf') {
        steps {
          githubNotify status: 'PENDING', context: 'Pipeline'
            runFunc()
        }
      }
    }
  }

Then runFunc is mocked as expected.

I tried newest version of JenkinsPipelineUnit: 1.13

This is a problem for me as I have very rich local functions I want to mock during pipeline testing. I do not want to mock functions like doSomeStuff in the example.
If this is a bug - do you know any workaround for this issue ?

[nestoracunablanco]

Hi @grzegorzgrzegorz I've taken a look at this issue, I found it interesting :)
The problem seems to be somewhere in the interceptClassMethods method in the InterceptingGCL class. A pull request will come in the next couple of minutes. A workaround until is released could be:
helper.registerAllowedMethod("runFunc", [], {})

[grzegorzgrzegorz]

Thanks for the tip @nestoracunablanco. This workaround solves the problem for function without arguments. However, my pipeline actually contains function with 2 arguments:
runFunc("p1", "p2")
in such a case I am unable to mock it and these attempts are running real runFunc with doSomeStuff function :
helper.registerAllowedMethod("runFunc", [String, String], {})
helper.registerAllowedMethod("runFunc", [String.class, String.class], {})
It is unclear to me which of above invocations is correct. Maybe all are wrong ? Could you please comment on this scenario as well?

[nestoracunablanco]

Hi @grzegorzgrzegorz I just made a change in order to be compatible with functions with more arguments. Since it uses dynamic compilation, is less performant, so let's check what @nre-ableton says about it.

[nre-ableton]

I'm not sure of the use-case here, but I don't see mocking local functions as something that would be a widely-used feature in JenkinsPipelineUnit. I would suggest either mocking the calls inside these functions (doSomeStuff, in the case of your example), or breaking out these functions to libraries where they can be easily mocked.

That said, I reviewed #510 independent of this discussion.

[grzegorzgrzegorz]

Hi, I have very different pipelines. Some of them have local functions, some of them use libraries. This is valid use case as sometimes there is no point in turning function into library if it is not used anywhere else. Sometimes, there are local functions which should become libraries but this is TODO status - I have to have working tests for such functions.

Anyway, in the spring I created my own testing framework as I just need to have tool which is useful for me. It took ma around 1 month but now I have full control of the functionality and so on... Using metaclass mocking and binding is sufficient to mock everything I need: steps, functions, objects, sections, properties, loading libraries, testing libraries. I also have something which I called emulators: aside of mocking, I can emulate steps and sections to work during the test like if they were run in Jenkins: SH, CHECKOUT etc. with default or user defined implementation. Workspace is also supported.

So to all who need to test your pipeline: try to start writing your own simple framework as Groovy metaprogramming is not so difficult. You will then decide for yourself which use case is valid and it will always suite your needs best.

[lemeurherve]

@grzegorzgrzegorz do you intend to publish your framework by any chance? Curious to see it for potential inspiration 🙂

[mcascone]

I have what I think is a similar problem/question. It has stood in my way for years at this point.

Say I have a class in vars/:

// vars/myFunction.groovy
def call() {
	childCall() 
}

def childCall() {
	// do stuff that i don't want to unit test
}

In my JPU tests, I can not find a way to mock childCall(). It will always run the real code.

class myFunctionTests extends BasePipelineTest {
  def myFunction

  @Before
  void setUp() {
    super.setUp()
    myFunction = loadScript("vars/myFunction.groovy")

	// none of these work
    helper.registerAllowedMethod('childCall', [], {})
	helper.registerAllowedMethod('myFunction.childCall', [], {})
	helper.registerAllowedMethod('call.childCall', [], {})

 @Test
 void myTest () {
	myFunction()

	assert stuff
  }
}

No matter how i try to mock childCall(), it always runs the real code, and i run into problems when that inner code exectutes.

One thing I have noticed is in the stack trace, the childCall is not listed:

  myFunction.call()
      myFunction.aSuccessfullyMockedExternalFunction()
	 // childCall() should be listed here, but it's not, and instead childCall's real code is executed
      pipelineBuild.aFunctionInsideChildCall()
myFunctionTests: localFunction: FAILURE

Is there any way to mock functions that are defined in the same vars/ code?

Thanks in advance!

[nestoracunablanco]

Hi @mcascone I just took a look at this issue. This is somewhat interesting to me :)
While I am taking a look at the code, here is a workaround that applies in your case:

class myFunctionTests extends BasePipelineTest {
  def myFunction

  @Before
  void setUp() {
    super.setUp()
    helper.registerAllowedMethod('childCall', [], {})
    helper.registerAllowedMethod('myFunction.childCall', [], {})
    helper.registerAllowedMethod('call.childCall', [], {})
    myFunction = loadScript("vars/myFunction.groovy")
}

 @Test
 void myTest () {
	myFunction()

	assert stuff
  }
}

I will let you know if I find a working solution.

[nestoracunablanco]

Making a quick analysis, I think I know the root cause of the problem. InterceptingGCL defines the methodInterceptor.
This methodInterceptor will be called every time a method is missing or invokeMethod class is called. But here is the catch: invokeMethod is not always called when a method is invoked. The language developers provide a solution for it, implementing the GroovyInterceptable interface, but as far as my investigation went the script class do not implement it by default.
Given the circumstances, I see for now this options here:

• Check if this can be fixed at language level implementing this GroovyInterceptable interface in the Script class.
• In registerAllowedMethod: calling interceptClassMethods immediately after. (But I ignore the side effects it could cause).

[mcascone]

Thanks for this @nestoracunablanco, I'm not sure how to read the bits about changing it in the JPU library, but I'll try the workaround in my tests ASAP!

[mcascone]

@nestoracunablanco, a related question: is there a way to isolate the childCall function, to only unit test that? In other words, to not mock it out, but to specifically call that childCall method so we can run unit tests on it. Thanks in advance!

[grzegorzgrzegorz]

@grzegorzgrzegorz do you intend to publish your framework by any chance? Curious to see it for potential inspiration slightly_smiling_face

Yes @lemeurherve I created very short example recently to show general idea:
https://github.com/grzegorzgrzegorz/pipeline-testing/tree/master
and described it here:
https://forseti79.e-kei.pl/wordpress/write-your-own-pipeline-testing-framework/

[mcascone]

Hi @nestoracunablanco, i'm finally getting around to trying this out, and I'm not having success. In this case my "parent" function is pipelineBuild, defined as call in vars/pipelineBuild.groovy, and it contains a child function called innerBuild. I can not find a way to mock innerBuild.

Note: callCounter is a neat thing i found somewhere to validate that the function you want has in fact been called, x number of times.

vars/pipelineBuild.groovy:

def call() {
  if(someLogic) { 
    innerBuild(buildParms)
  }
  else {
    echo "No more builds to do"
  }
}

def innerBuild (HashMap buildInfo) {
  // ... a bunch of stuff ...
}

test/com/myCompany/pipelineBuildTests.groovy:

class pipelineBuildTests extends BasePipelineTest {
  def pipelineBuild

  def callCounter (String methodName) {
    return (helper.callStack.findAll { it?.methodName == methodName })?.size()
  }

  @Before
  void setUp() {
    super.setUp()
    pipelineBuild = loadScript("vars/pipelineBuild.groovy")
  }

  @Test
  void runsInnerbuild () {
    helper.registerAllowedMethod('innerBuild', [HashMap], {})
    helper.registerAllowedMethod('pipelineBuild.innerBuild', [HashMap], {})
    helper.registerAllowedMethod('call.innerBuild', [HashMap], {})
    pipelineBuild()

    assert 1 == callCounter('innerBuild')
  }
}

I tried it with the mocks in the test declaration as shown; as well as before and after the loadScript line. Regardless of the mocks' location, i still get this error every time:

groovy.lang.MissingMethodException: No signature of method: pipelineBuild.folderCreateOperation() is applicable for argument types: (String) values: [Release-Packages]

and that error means the test is actually calling innerBuild instead of mocking it.

Is there something I'm doing wrong? Or is this just a (huge) limitation of the JPU framework?
Thanks!

[nre-ableton]

Because you are mocking singletons, things will be a bit trickier. But normally, the approach would look something like this:

class pipelineBuildTests extends BasePipelineTest {
  Object script

  class MockPipelineBuild {
    def innerBuild (HashMap buildInfo) {
      // ... a bunch of stuff ...
    }
  }

  @Before
  void setUp() {
    super.setUp()
    script = loadScript("vars/empty.groovy")  // NOTE: Load an empty pipeline context here
    script.pipelineBuild = new MockPipelineBuild()
  }

However, it's very tricky because your singleton is acting as both the pipeline context and the thing you want to test. Groovy, as flexible as it is, is not a duck-typed language which you can just redirect and mock on the fly.

In general, we recommend using classes that contain the build logic and thin singletons that just act as passthru layers. Generally speaking, you shouldn't need to test your singletons at all, because trying to do so usually ends up being a giant mess (especially when you have singletons calling functions in other singletons).

Please refer to this project's documentation on the recommended approach to writing testable libraries.

[mcascone]

Hi, thanks for this. I know I'm "cheating" in a way using primarily singletons in vars/ as my groovy classes. I have tried several times and just have never been able to make things work using classes defined in src/. I know a lot more about, well, everything, since last I tried, so it's probably time to try again.

Thanks!
max

[grzegorzgrzegorz]

It is not a crime to work with libraries in this way, so it is not cheating. Jenkins documentation describes it here: https://www.jenkins.io/doc/book/pipeline/shared-libraries/#defining-custom-steps so people are using it.
Testing framework should just support it.

[nre-ableton]

@grzegorzgrzegorz Nobody ever said it was a "crime". It's just not an ideal approach to writing testable code.

Similarly, it's not a crime to use global state in a program, or to put all of your logic in one giant, monolithic class. Both of these things are allowed by software tools, but it will just cause more headaches when trying to write tests.

For example, in the documentation that you linked to, documents the @Grab step but then advises against using it. They are defining a best practice -- this project is simply doing the same. 🙂

[mcascone]

Is there any way i could have my pipeline reviewed by others, maybe you folks on this thread, or other Jenkins experts? It might be hard to obfuscate every bit of company-private info, but there isn't much that would be a security concern.

[nre-ableton]

@mcascone Feel free to submit a draft PR on whatever repository and link it here. Likewise, you could create a GitHub gist with an anonymized pipeline and I'd be happy to critique it.

[mcascone]

@nre-ableton thanks for the offer! It's a whole library repo so it'd take more than a gist... i suppose I could set my public github as another remote from it, and push there. I'll see what I can do!

[nre-ableton]

@mcascone Thanks, I just got the invite. It's getting a bit late in my timezone, so I'll check out the repo next week. 👍

[mcascone]

Hey, thanks a lot! FYI, the main pipeline template is at vars/servicePipeline.groovy. There's a link to a confluence wiki with the docs, but I don't think I can give you access to it as it's my company's instance. Maybe I can spit a PDF out from confluence or something.

…

On Fri, Oct 7, 2022 at 10:26 AM Nik Reiman ***@***.***> wrote: @mcascone <https://github.com/mcascone> Thanks, I just got the invite. It's getting a bit late here, so I'll check out the repo next week. 👍 — Reply to this email directly, view it on GitHub <#461 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7S5ZYUO44AIE4DB4UQGGDWCA6KVANCNFSM5KS62IPQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Thanks, Max