Make utilities available for Jupyter Users too

[koaning]

Some of the tools we provide are really swell for streamlit, but some of use like to hack. We should also make some of the charts/tools available from Jupyter.

Something like;

from rasalit import load_interpreter
from rasalit.charts import nlu_chart

[tanmaylaud]

Hi @koaning, I would like to contribute here. I have been looking to learn how to create such iPy visuals/widgets

[koaning]

The tools that I'm interested in porting here are mainly the Altair charts for now. I've got one bulk labelling demo in a notebook that uses widgets but that's already made. I'm mainly interested in porting the spelling/nlu-live charts for now. Once ported I'd also prefer to use these implementations from streamlit.

[koaning]

I might ask another question though since you mentioned you were interested, what would you like to make that would be useful in your day-to-day work in NLP? 😄

[tanmaylaud]

The idea would be to provide a visual for any pipeline, especially the annotation-based pipelines to show what's being annotated and how ( NER, Question Answering ) similar to displacy.
Another could be model attention visualisation to allow interpretation of models

[koaning]

I knew that displacy does NER, but I was unaware of question answering. Do you have a link?

I'm working on a model attention visualization already but that's something that I'm building for our research team internally. It'll be open-sourced but only after my colleagues confirm it is useful.

If you're working on a variant of a labeling tool for the notebook you might want to check if you can extend pigeon.

[tanmaylaud]

For QnA ( tagging a span) , I have used displacy's manual annotation function where you just pass a dict with start, end index params.
For streamlit based apps, I have used st-annotate :
https://github.com/deepset-ai/haystack/tree/master/ui

[koaning]

@tanmaylaud for my understanding, are you familiar with Rasa?

[tanmaylaud]

@tanmaylaud for my understanding, are you familiar with Rasa?

@koaning I am familiar with Rasa and what is used for and the methodologies ( thanks to the videos on Rasa Youtube channel), but unfortunately I haven't used it yet for any task, so I won't know the internals

[koaning]

The best thing for Rasalit would be to ask ourselves: what chart would we like to see? Maybe there's a view missing that might make it easier to folks to understand why certain predictions are being made. When you consider this question, is there a view/chart that comes to your mind?

[tanmaylaud]

@koaning I am guessing Rasa uses some form of beam search. In that case a minimum span tree could be created and then the chosen beam could be highlighted.
And how a change in params could lead to another beam selection