IndexError when new value appears in postcheck

[jaakub]

Description of Issue/Question

When a device is upgraded, a new environmental item is detected in post check, whilst not detected in precheck, the test errors out.

ERROR!! AttributeError 
Complete Message: 'NoneType' object has no attribute 'findtext'

Setup

config.yaml

hosts:
  - device: 192.168.0.1
    username: myuser
    passwd: mypassword
    port: 22
tests:
  - test.yaml

test.yaml

chassis_environment:
  - command: show chassis environment
  - iterate:
      xpath: ./environment-item
      id: ./name
      tests:
        - no-diff: ./status

Precheck XML: https://snippi.com/s/2ftc9mu
Postcheck XML: https://snippi.com/s/8k1k71g

Steps to Reproduce Issue

Run compare:
jsnapy --check pre post -f config.yaml

Receive the following error due to FPC 5 PLX PCIe Switch TSen & FPC 7 PLX PCIe Switch TSen are missing in the precheck.

********************* Command: show chassis environment *********************
['FPC 5 PLX PCIe Switch TSen'] status has changed. Post - OK | Pre - None. Comment: None.
['FPC 5 PLX PCIe Switch Chip'] status has changed. Post - OK | Pre - None. Comment: None.
ERROR!! AttributeError 
Complete Message: 'NoneType' object has no attribute 'findtext'
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/home/jakub/.venv/lib64/python3.6/site-packages/jnpr/jsnapy/jsnapy.py", line 664, in connect
    **kwargs)
  File "/home/jakub/.venv/lib64/python3.6/site-packages/jnpr/jsnapy/jsnapy.py", line 487, in get_test
    **kwargs)
  File "/home/jakub/.venv/lib64/python3.6/site-packages/jnpr/jsnapy/jsnapy.py", line 416, in compare_tests
    post_snap_file)
  File "/home/jakub/.venv/lib64/python3.6/site-packages/jnpr/jsnapy/check.py", line 778, in generate_test_files
    action)
  File "/home/jakub/.venv/lib64/python3.6/site-packages/jnpr/jsnapy/check.py", line 444, in compare_reply
    final_boolean_expr = self.expression_builder(testcases, None, **kwargs)
  File "/home/jakub/.venv/lib64/python3.6/site-packages/jnpr/jsnapy/check.py", line 325, in expression_builder
    last_test_instance = kwargs['op'].test_details[kwargs['teston']][-1]
IndexError: list index out of range

/var/logs/jsnapy/jsnapy.log:

2022-03-15 09:16:28,442 - jnpr.jsnapy.jsnapy - DEBUG - 
 �[34mjsnapy.cfg file location used : /home/jakub/.venv/etc/jsnapy
2022-03-15 09:16:28,442 - jnpr.jsnapy.jsnapy - DEBUG - 
 �[34mConfiguration file location used : /home/jakub/juniper-jsnapy/configs
2022-03-15 09:16:28,459 - jnpr.jsnapy.check - INFO - 
 �[34m****************************** Device: bdr1.jul ******************************
2022-03-15 09:16:28,459 - jnpr.jsnapy.check - INFO - 
 Tests Included: chassis_environment 
2022-03-15 09:16:28,459 - jnpr.jsnapy.check - INFO - 
 �[34m********************* Command: show chassis environment *********************
2022-03-15 09:16:28,462 - jnpr.jsnapy.operator - DEBUG - 
 �[34m----------------------Performing no-diff Test Operation----------------------
2022-03-15 09:16:28,468 - jnpr.jsnapy.operator - DEBUG - 
 ['Routing Engine 1 CPU']
2022-03-15 09:16:28,469 - jnpr.jsnapy.operator - DEBUG - 
 ['FPC 4 XL_XR1 TSen']
2022-03-15 09:16:28,469 - jnpr.jsnapy.operator - DEBUG - 
 ['CB 1 Exhaust A']
2022-03-15 09:16:28,470 - jnpr.jsnapy.operator - DEBUG - 
 ['FPC 7 Exhaust A']
2022-03-15 09:16:28,471 - jnpr.jsnapy.operator - DEBUG - 
 ['FPC 7 XM Chip']
2022-03-15 09:16:28,472 - jnpr.jsnapy.operator - DEBUG - 
 ['Top Fan Tray Temp']
2022-03-15 09:16:28,473 - jnpr.jsnapy.operator - DEBUG - 
 ['FPC 7 XF TSen']
2022-03-15 09:16:28,473 - jnpr.jsnapy.operator - DEBUG - 
 ['FPC 0 MQ 1 Chip']
2022-03-15 09:16:28,474 - jnpr.jsnapy.operator - DEBUG - 
 ['FPC 4 XL_XR1 Chip']
2022-03-15 09:16:28,474 - jnpr.jsnapy.operator - ERROR - 
 �[31mERROR!! AttributeError 
Complete Message: 'NoneType' object has no attribute 'findtext'

Versions Report

(.venv) $ jsnapy --version
JSNAPy version: 1.3.6
(.venv) $ python3 --version
Python 3.6.8

[ddivins]

I am checking route next-hops. When a new route appears in post I get an error trying to reference a related element. The first test case has no issue, just looking at routes. When I look at next-hops and reference the route, is when I run into issues

test.yaml

tests_include:
  - test_command_routes

test_command_routes:
  - command: show route
  - iterate:
      xpath: '/route-information/route-table/rt'
      id: rt-destination
      tests:
         - no-diff: rt-destination
           err: "Test Failed!! rt-destination got changed, before it was <{{pre['rt-destination']}}>
                 NOW it is <{{post['rt-destination']}}>"
           info: "Test Passed!! rt-destination is same, before it is <{{pre['rt-destination']}}>
                  NOW it is <{{post['rt-destination']}}>"

  - iterate:
      xpath: '/route-information/route-table/rt'
      id: rt-destination
      tests:
          - no-diff: rt-entry/nh/to
            err: "Test Failed!! Next-Hop changed, before it was <{{pre['rt-destination']}}> via <{{pre['rt-entry/nh/to']}}>
                  NOW it is via <{{post['rt-entry/nh/to']}}>"
            info: "Test Passed!! Next-Hop is same, before it was <{{pre['rt-destination']}}> via <{{pre['rt-entry/nh/to']}}>
                   NOW it is still via <{{post['rt-entry/nh/to']}}>"

output:

**************************** Device: 192.168.3.4 ****************************
Tests Included: test_command_routes 
**************************** Command: show route ****************************
----------------------Performing no-diff Test Operation----------------------
Test Passed!! rt-destination is same, before it is <['192.168.3.0/24']> NOW it is <['192.168.3.0/24']>
Test Passed!! rt-destination is same, before it is <['ff02::2/128']> NOW it is <['ff02::2/128']>
Test Passed!! rt-destination is same, before it is <['192.168.3.4/32']> NOW it is <['192.168.3.4/32']>
Test Passed!! rt-destination is same, before it is <['0.0.0.0/0']> NOW it is <['0.0.0.0/0']>
Test Failed!! rt-destination got changed, before it was <None> NOW it is <216.12.128.1/32>
FAIL | All "rt-destination" is not same in pre and post snapshot [ 4 value matched / 1 value failed ]
----------------------Performing no-diff Test Operation----------------------
Test Passed!! Next-Hop is same, before it was <192.168.3.0/24> via <None> NOW it is still via <None>
Test Passed!! Next-Hop is same, before it was <ff02::2/128> via <None> NOW it is still via <None>
Test Passed!! Next-Hop is same, before it was <192.168.3.4/32> via <None> NOW it is still via <None>
Test Passed!! Next-Hop is same, before it was <0.0.0.0/0> via <['192.168.3.1']> NOW it is still via <['192.168.3.1']>
ERROR!! AttributeError 
Complete Message: 'NoneType' object has no attribute 'findtext'
------------------------------- Final Result!! -------------------------------
test_command_routes : Failed
Total No of tests passed: 0
Total No of tests failed: 2 
Overall Tests failed!!! 

Should this fail the second test but at least tell me what the post route element was that is new? Or report the missing pre element as None?
Also, if I had more routes would this continue to process after that error?

Sorry if my test case is a mess, I'm learning :-)

Attached are my pre and post snapshots (renamed .txt so they can paste)

192.168.3.4_post_show_route.txt
192.168.3.4_pre_show_route.txt

Versions:

(venv) ddivins@ddivins-mbp jsnap % jsnapy --version
JSNAPy version: 1.3.7
(venv) ddivins@ddivins-mbp jsnap % python3 --version
Python 3.10.4

[chidanandpujar]

issue is fixed and working fine now, we will close this issue as fixed .

jsnapy --check pre post -f common_config.yml -v
jsnapy.cfg file location used : /root/jsnapy_test1/venv/etc/jsnapy
Configuration file location used : /root/jsnapy_test1/venv/etc/jsnapy
**************************** Device: x.x.x.x ****************************
Tests Included: test_command_routes 
**************************** Command: show route ****************************
----------------------Performing no-diff Test Operation----------------------
Test Passed!! rt-destination is same, before it is <['0.0.0.0/0']> NOW it is <['0.0.0.0/0']>
Test Failed!! rt-destination got changed, before it was <None> NOW it is <216.12.128.1/32>
Test Passed!! rt-destination is same, before it is <['192.168.3.4/32']> NOW it is <['192.168.3.4/32']>
Test Passed!! rt-destination is same, before it is <['ff02::2/128']> NOW it is <['ff02::2/128']>
Test Passed!! rt-destination is same, before it is <['192.168.3.0/24']> NOW it is <['192.168.3.0/24']>
FAIL | All "rt-destination" is not same in pre and post snapshot [ 4 value matched / 1 value failed ]
----------------------Performing no-diff Test Operation----------------------
Test Passed!! Next-Hop is same, before it was <0.0.0.0/0> via <['192.168.3.1']> NOW it is still via <['192.168.3.1']>
Test Failed!! Next-Hop changed, before it was <None> via <None> NOW it is via <192.168.3.1>
Test Passed!! Next-Hop is same, before it was <192.168.3.4/32> via <None> NOW it is still via <None>
Test Passed!! Next-Hop is same, before it was <ff02::2/128> via <None> NOW it is still via <None>
Test Passed!! Next-Hop is same, before it was <192.168.3.0/24> via <None> NOW it is still via <None>
FAIL | All "rt-entry/nh/to" is not same in pre and post snapshot [ 4 value matched / 1 value failed ]
------------------------------- Final Result!! -------------------------------
test_command_routes : Failed
Total No of tests passed: 0
Total No of tests failed: 2 
Overall Tests failed!!! 

Thanks
Chidanand

[chidanandpujar]

issue is fixed #399