Jsonpath transformation doesn't work with null values in json data.

[pacive]

Since commit f1c63ba7177bf007b457f60676c8701e87c74343 the jsonpath transformation no longer work properly when the json data contains null values. Instead of returning null it returns the whole json string when accessing a key with null value.

For example when trying to access the 'upgrade' field in
{"current":{"name":"8461R2","version":8461,"release":2},"upgrade":null}
the whole string is returned instead of null.

Steps to reproduce:

rule "test"
	when
		Item Test changed
	then
		var json1 = "{\"current\":{\"name\":\"8461R2\",\"version\":8461,\"release\":2},\"upgrade\":\"null\"}"
		var json2 = "{\"current\":{\"name\":\"8461R2\",\"version\":8461,\"release\":2},\"upgrade\":null}"
		logInfo("Test", transform("JSONPATH", "$.upgrade", json1))
		logInfo("Test", transform("JSONPATH", "$.upgrade", json2))
end

returns

2018-01-03 15:07:18.458 [INFO ] [.eclipse.smarthome.model.script.Test] - null
2018-01-03 15:07:18.465 [INFO ] [.eclipse.smarthome.model.script.Test] - {"current":{"name":"8461R2","version":8461,"release":2},"upgrade":null}

More info in this post on the OpenHAB forum.

[triller-telekom]

The change was indeed in this line as you mentioned in the linked openHAB forum post.

But the problem is a few lines above

Object transformationResult = JsonPath.read(source, jsonPathExpression);

We cannot tell whether the received value from the read call is really null or if there was an error.

[pacive]

I which cases does an error in the JsonPath.read() lead to a null value being returned? If I try to access a nonexistent field for example, it throws a PathNotFoundException which is handled and a TransformationException is thrown, instead of returning null as it previosly did.

rule "test"
	when
		Item Test changed
	then
		var json1 = "{\"current\":{\"name\":\"8461R2\",\"version\":8461,\"release\":2},\"upgrade\":\"null\"}"
		var json2 = "{\"current\":{\"name\":\"8461R2\",\"version\":8461,\"release\":2},\"upgrade\":null}"
		logInfo("Test", transform("JSONPATH", "$.upgrade", json1))
		logInfo("Test", transform("JSONPATH", "$.upgrade", json2))
                logInfo("Test", transform("JSONPATH", "$.someField", json2))
end

logs:

2018-01-09 15:57:10.505 [INFO ] [.eclipse.smarthome.model.script.Test] - null

2018-01-09 15:57:10.515 [INFO ] [.eclipse.smarthome.model.script.Test] - {"current":{"name":"8461R2","version":8461,"release":2},"upgrade":null}

2018-01-09 15:57:10.525 [ERROR] [ore.transform.actions.Transformation] - Error executing the transformation 'JSONPATH': Invalid path '$.someField' in '{"current":{"name":"8461R2","version":8461,"release":2},"upgrade":null}'

2018-01-09 15:57:10.534 [INFO ] [.eclipse.smarthome.model.script.Test] - {"current":{"name":"8461R2","version":8461,"release":2},"upgrade":null}

[triller-telekom]

Thank you for trying this out with an example. I am wondering this is always the case? I could not find any documentation for this behavior, did you come across any?

[pacive]

Just the JsonPath readme. It can be configured to return null if an exception occurs, but that isn't the case in this implementation, so i don't think itshould return null unless the requested field is null.

@kaikreuzer or @SJKA, any more input on this?

[triller-telekom]

Sounds like what we need. Do you want to come up with a PR for this?

[sjsf]

As you can see from the discussion in #4379, we were all a little unsure what to return in that case. With your analysis it indeed would make sense to define null as a valid outcome. If the element doesn't exist it will still throw the TransformationException.

It can't be the real null though, it has to be the string representation of UnDefType.NULL, i.e. "NULL"

[pacive]

That would solve my problem, just need to change null to "NULL" in my rule. Thanks @SJKA!