logging does not work when running jsnapy with ansible

[avocadonetwork1]

Description of Issue/Question

Hi,
I have integrated jsnapy with ansible so that I can use ansible playbook to run jsnapy.
However the info and err from my jsnapy tests are not outputing when I run the ansible playbook.
As the documentation suggests, I have tried both of the configs below on ansible.cfg but still no output
callback_enabled = juniper.device.jsnapy
callback_whitelist = juniper.device.jsnapy

Setup

The following files are created

1. snapcheck_playbook.yml (ansible playbook) ,
2. jsnapy_tests.yml (which tests I am running) and
3. test_snmp.yml (test itself)

I am adding the files below

`
snapcheck_playbook.yml

• name: "Ansible Jsnapy Snapcheck"
  hosts: all
  strategy: free
  connection: local
  gather_facts: false

  tasks:

  • name: Run Jsnapy check
    juniper.device.jsnapy:
    action: snapcheck
    config_file: jsnapy_tests.yml
    logfile: jsnapy_ansible.log
    register: result

  • name: Run Jsnapy check
    assert:
    that:

    • "result.passPercentage == 100"

jsnapy_tests.yml

tests:

• test_snmp.yml

test_snmp.yml

Test SNMP config:

• rpc: get-configuration
• item:
  xpath: /configuration/snmp
  tests:
  - exists: community[name='mycommunity']
  info: SNMP community ''mycommunity' is configured.
  err: SNMP community ''mycommunity' is not configured!
  `

Steps to Reproduce Issue

(Include debug logs if possible and relevant. Error trace would be helpful too)
To run the playbook:
$ ansible-playbook snapcheck_playbook.yml -l mydevice

PLAY [Ansible Jsnapy Snapcheck] *******************************************************************************************************

TASK [Run Jsnapy check] ***************************************************************************************************************
ok: [mydevice]

TASK [Run Jsnapy check] ***************************************************************************************************************
fatal: [mydevice]: FAILED! => {
"assertion": "result.passPercentage == 100",
"changed": false,
"evaluated_to": false,
"msg": "Assertion failed"
}

PLAY RECAP ****************************************************************************************************************************
mydevice : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

As you can see from the above, the tests failed as expected (because the snmp community is not set at my router yet. However the err message is not displayed on the output.
I can see the errors on the jsnapy.log but not on the output from the command

Versions Report

(Provided by running jsnapy --version. Please also mention python version.)
$ jsnapy --version
JSNAPy version: 1.3.6

$ ansible --version
ansible [core 2.11.1]

$ python --version
Python 3.9.5

[chidanandpujar]

issue is reproducible with the following information

cat snapcheck_playbook.yml 
---
- name: Test juniper.device.jsnapy module
  hosts: all
  gather_facts: false
  tasks:
##################################################
####    TEST 2                                  ##
##################################################
    - name: "TEST 2 - Execute SNAPCHECK "
      juniper.device.jsnapy:
        config_file: jsnapy_tests.yml
        action: snapcheck
      register: result 

    - name: Verify JSNAPy tests passed
      assert:
        that:
          - "result.passPercentage == 100"

cat jsnapy_tests.yml 
tests:
  test_snmp.yml

 cat test_snmp.yml 
tests_include:
  - test_snmp_config 

test_snmp_config:
  - rpc: get-configuration
  - iterate:
      xpath: /configuration/snmp
      tests:
        - exists: community[name='mycommunity'] 
          info: SNMP community ''mycommunity' is configured. 
          err: SNMP community ''mycommunity' is not configured!

jsnapy --snapcheck -f jsnapy_tests_with_host.yml
Connecting to device x.x.x.x ................
Taking snapshot of RPC: get-configuration
**************************** Device: x.x.x.x ****************************
Tests Included: test_snmp_config
RPC is get-configuration
ERROR!! Nodes are not present in given : </configuration/snmp>
FAIL | All "community[name='mycommunity']" do not exists at xpath "/configuration/snmp" [ 0 value matched / 1 value failed ]
------------------------------- Final Result!! -------------------------------
test_snmp_config : Failed
Total No of tests passed: 0
Total No of tests failed: 1
Overall Tests failed!!!

ansible-playbook snapcheck_playbook.yml

PLAY [Test juniper.device.jsnapy module] **********************************************************************************************************************************

TASK [TEST 2 - Execute SNAPCHECK] *****************************************************************************************************************************************
ok: [local_connection_testcases_mode_ssh]

TASK [Verify JSNAPy tests passed] *****************************************************************************************************************************************
fatal: [local_connection_testcases_mode_ssh]: FAILED! => changed=false
assertion: result.passPercentage == 100
evaluated_to: false
msg: Assertion failed

PLAY RECAP ****************************************************************************************************************************************************************
local_connection_testcases_mode_ssh : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

Thanks
Chidanand