nan in Perplexity for big file with train data

[Guince]

Hi!

I use bigartm=0.9.2 to train my topic model. And I run into some unexpectable behaviour of Perplexity score. I have a very big file with train data. And I got nan in perplexity score after train my model. I find out that if train file is bigger than 1000 documents, I have nan, and when it <= I got numerical value of Perplexity score. I tried to use bigartm==0.10.2, it doesn't help.

I also tried to use batch_size=100, it doesnt't help too. But when I set batch_size==100000 or batch_size==10000, I have numerical value of Perplexity score.

I happy that I solved my problem, but this behaviour is unexpectable. And now it turns out that I should use very big batch_size. My train data is getting biggerover time, so I want to know in advance what batch_size I shold use to not have nan in Perplexity score in future. Which batch_size do you recommend? Should I use batch_size that is the same size as number of documents in my train collection?

[Guince]

Collection of 1000 documents is just an example. My collection is much bigger.

batch_size=10 000 didn't solve the problem for my train data.