How to reduce the output to a minimum?

[haimivan]

Hi,

my use case for traceback_with_variables is mainly during development time. Therefore I just need a small overview what was wrong with the variables that actually caused the exception.

My question is: how can I customize it that the extra output that traceback_with_variables generates only comprises the very variables that lead to the exception?

Example:

Without using it, for this:

# from traceback_with_variables import activate_by_import
def main():
    n = 0
    x = 15
    print(1 / n)

main()

I get:

/home/usesr/venv/numba/bin/python3.6 /home/usesr/PycharmProjects/myProject/attic04.py
Traceback (most recent call last):
  File "/home/usesr/PycharmProjects/myProject/attic04.py", line 7, in <module>
    main()
  File "/home/usesr/PycharmProjects/myProject/attic04.py", line 5, in main
    print(1 / n)
ZeroDivisionError: division by zero

Process finished with exit code 1

With using it, I get:

/home/usesr/venv/numba/bin/python3.6 /home/usesr/PycharmProjects/myProject/attic04.py
Traceback with variables (most recent call last):
  File "/home/usesr/PycharmProjects/myProject/attic04.py", line 7, in <module>
    main()
      __name__ = '__main__'
      __doc__ = None
      __package__ = None
      __loader__ = <_frozen_importlib_external.SourceFileLoader object at 0x7f78c2d32f28>
      __spec__ = None
      __annotations__ = {}
      __builtins__ = <module 'builtins' (built-in)>
      __file__ = '/home/usesr/PycharmProjects/myProject/attic04.py'
      __cached__ = None
      activate_by_import = <module 'traceback_with_variables.activate_by_import' from '/home/usesr/venv/numba/lib/python3.6/site-packages/traceback_with_variables/activate_by_import.py'>
      main = <function main at 0x7f78c2d58e18>
  File "/home/usesr/PycharmProjects/myProject/attic04.py", line 5, in main
    print(1 / n)
      x = 15
      n = 0
builtins.ZeroDivisionError: division by zero

Process finished with exit code 1

Would it be possible to configure it (at one place in my code) that I only get this output:

/home/usesr/venv/numba/bin/python3.6 /home/usesr/PycharmProjects/myProject/attic04.py
Traceback (most recent call last):
  File "/home/usesr/PycharmProjects/myProject/attic04.py", line 7, in <module>
    main()
  File "/home/usesr/PycharmProjects/myProject/attic04.py", line 5, in main
    print(1 / n)
    n = 0
ZeroDivisionError: division by zero

Process finished with exit code 1

(so:
n = 0
lead to the exception, but
x = 15
was not part of the game.

As I have a lot of variables in my code, I get overwhelmed by the output and it takes me also time to find the value of the variable in the output that was causing the exception.

If I just could use a different import, that would be perfect for me :-) )

[andy-landy]

Hi, thanks for the feedback! I'm in the middle of implementing exactly this feature :) I happened to change signatures with some previous commits, so I'll have to increase major version to 2, so I want to accumulate all planned features at once in order to avoid another major version increase ;) I think I'll be done in a week.

[andy-landy]

Fixed in 2.0.0, now you can customly skip variables. activate_by_import makes it skip globals.

[haimivan]

Thanks a lot for your effort and the new release. It looks very good!!! 👍

Is it maybe also possible to limit the output to these variables that were really involved in the exception?

(so in my example above the variable n caused the exception, but x did not)

For me it would be perfect, if the x would not appear in the print-out at all (the x in my project stands for more than 40 other variables, and more than half of them are pandas-DataFrames and numpy arrays. This results in a very long list of variables in this case, and I have to scroll through all of them).

[andy-landy]

Yes, that's nice! I added it to the feature list. Thanks for the idea!

[dvir-siga]

Great package, thanks for releasing!

I'm also dealing with numpy & pandas and want more control over the output.

I suggest allowing the user to define a callback which will be called for every variable, to handle if & how to print it.

This is only a partial solution but should be simple enough to implement and allows almost unlimited control.

I imagine it's possible to pass a detailed dict as kwargs, maybe with the following keys:
var_name, var_value, var_type?, file, line, module, line_string?, exception

The user will be able to print whatever is needed and return a boolean value specifying if to print the default output for the variable.

This will support multiple functionalities, e.g.:

def callback(var_name, line_string, **kwargs):
  if var_name not in line_string:
    return False
  return True

def callback(var_name, var_value, **kwargs):
  if isinstance(var_value, np.ndarray) or isinstance(var_value, pd.DataFrame):
    print(var_name, 'shape:', shape)
    if var_value.size>100:
      return False
  return True

[andy-landy]

Hi, please take a look at https://github.com/andy-landy/traceback_with_variables/blob/master/examples/format_customized.py at custom_var_printers. Does it fit your use case?

[dvir-siga]

custom_var_printers looks great, I somehow missed it.

I would suggest adding more arguments to give even more control, e.g. line_string above could bring us closer to @haimivan 's goal.

But it's already good enough for me, thanks!