Maximum Text Length Translations

[Alexbuchwald]

Hi!
Im new to Spark NLP and Im working on translation tasks. I noticed that the pretrained models work really well, but unfortunately text will be cut off after a certain length. I was looking through the documentations, but I could not find a refrence or setting for the maximum length. Do you have an idea how to handle very long, multi-sentence long text data?

Thanks in advance! :)

[maziyarpanahi]

Hi,

You are using NLU: https://github.com/JohnSnowLabs/nlu

In Spark NLP there are 2 parameters to set the input/output lengths. Also, you can feed sentences and which breaks the documents to shorter sentences for better and faster translation.

Perhaps @C-K-Loan can share more info about how to do these in NLU.

[Alexbuchwald]

Thanks