Improve error reporting of interpreter fallback

[triceo]

jpyinterpreter is a very complex piece of code. It would be naive to think that it implements every little nuance of Python bytecode correctly. (Case in point: #146.)

The current behavior is that such issues throw an exception, and the user is effectively locked out of using OptaPlanner. I am proposing that we implement a different behavior instead - we log an error, and fall back to unoptimized version. Slower performance beats no performance 100 % of the time.

[Christopher-Chianelli]

In fact, we do that, see

optapy/jpyinterpreter/tests/test_functions.py

Lines 266 to 286 in 3c35ef7

	def test_code_works_if_compilation_failed():
	def my_function(x: int) -> tuple:
	def inner_function(y: int) -> type:
	nonlocal x
	class MyClass: # TODO: Replace this with something else that fails when class creation is supported
	def __init__(self):
	self.outer_arg = x
	self.inner_arg = y
	def get_args(self):
	return self.outer_arg, self.inner_arg
	return MyClass
	return inner_function(2 * x)().get_args()
	verifier = verifier_for(my_function)
	verifier.verify(1, expected_result=(1, 2))
	verifier.verify(2, expected_result=(2, 4))
	verifier.verify(3, expected_result=(3, 6))

and

optapy/jpyinterpreter/src/main/python/python_to_java_bytecode_translator.py

Lines 227 to 300 in 3c35ef7

	elif inspect.iscode(value):
	try:
	from org.optaplanner.jpyinterpreter.types import PythonLikeFunction, PythonCode
	java_class = translate_python_code_to_java_class(value, PythonLikeFunction)
	out = PythonCode(java_class)
	put_in_instance_map(instance_map, value, out)
	return out
	except:
	from org.optaplanner.jpyinterpreter.types import PythonLikeFunction, PythonCode
	java_class = translate_python_code_to_python_wrapper_class(value)
	out = PythonCode(java_class)
	put_in_instance_map(instance_map, value, out)
	return out
	elif type(value) is object:
	java_type = type_to_compiled_java_class[type(value)]
	out = CPythonBackedPythonLikeObject(java_type)
	put_in_instance_map(instance_map, value, out)
	CPythonBackedPythonInterpreter.updateJavaObjectFromPythonObject(out,
	JProxy(OpaquePythonReference, inst=value,
	convert=True),
	instance_map)
	return out
	elif not inspect.isfunction(value) and type(value) in type_to_compiled_java_class:
	if type_to_compiled_java_class[type(value)] is None:
	return None
	java_type = type_to_compiled_java_class[type(value)]
	if isinstance(java_type, CPythonType):
	return None
	java_class = java_type.getJavaClass()
	out = java_class.getConstructor(PythonLikeType).newInstance(java_type)
	put_in_instance_map(instance_map, value, out)
	CPythonBackedPythonInterpreter.updateJavaObjectFromPythonObject(out,
	JProxy(OpaquePythonReference, inst=value,
	convert=True),
	instance_map)
	if isinstance(out, AbstractPythonLikeObject):
	for (key, value) in getattr(value, '__dict__', dict()).items():
	out.setAttribute(key, convert_to_java_python_like_object(value, instance_map))
	return out
	elif inspect.isbuiltin(value) or is_c_native(value):
	return None
	elif inspect.isfunction(value):
	try:
	from org.optaplanner.jpyinterpreter.types import PythonLikeFunction
	wrapped = PythonLikeFunctionWrapper()
	put_in_instance_map(instance_map, value, wrapped)
	out = translate_python_bytecode_to_java_bytecode(value, PythonLikeFunction)
	wrapped.setWrapped(out)
	put_in_instance_map(instance_map, value, out)
	return out
	except:
	return None
	else:
	try:
	java_type = translate_python_class_to_java_class(type(value))
	if isinstance(java_type, CPythonType):
	return None
	java_class = java_type.getJavaClass()
	out = java_class.getConstructor(PythonLikeType).newInstance(java_type)
	put_in_instance_map(instance_map, value, out)
	CPythonBackedPythonInterpreter.updateJavaObjectFromPythonObject(out,
	JProxy(OpaquePythonReference, inst=value,
	convert=True),
	instance_map)
	if isinstance(out, AbstractPythonLikeObject):
	for (key, value) in getattr(value, '__dict__', dict()).items():
	out.setAttribute(key, convert_to_java_python_like_object(value, instance_map))
	return out
	except:
	return None

. I suspect it falls through because of

optapy/jpyinterpreter/src/main/python/python_to_java_bytecode_translator.py

Lines 1220 to 1224 in 3c35ef7

	out = PythonClassTranslator.translatePythonClass(python_compiled_class)
	type_to_compiled_java_class[python_class] = out
	PythonClassTranslator.setSelfStaticInstances(python_compiled_class, out.getJavaClass(), out,
	CPythonBackedPythonInterpreter.pythonObjectIdToConvertedObjectMap)
	return out

, which is not wrapped in a try...except (and hence why it is caused by optapy annotations which directly call translate_python_class_to_java_class to get the superclass needed)

[triceo]

Thanks for the pointers. They don't show me the error messages that we output if/when this happens, though. (We want people to tell us of these errors, don't we? Even if we handle them properly by falling back.)

[Christopher-Chianelli]

The error situation can definitely improve. I am planning in the future to return a CompliationResult instead of the raw Java class. The CompliationResult would have fields for errors and warnings (ex: potential None reference, type mismatch, missing method/field, etc). The translator can then report not just mistranslation errors (our errors) but also user errors.

[triceo]

Ok, I updated the title of this issue.