VideoReader initialization becomes very slow after loading many videos

[alexnwang]

Decord installed with pip, observed usingVideoReader alongside the Pytorch dataloader for decoding videos.
Code looks something like:

class Dataset
     def __getitem__(self, idx):
          vr = VideoLoader(self.video_paths[idx], num_threads=1, device=cpu(0))
          images = vr.get_batch([...]).asnumpy()
          return images

However, after some number of examples, (dependent on num_workers), creating the VideoLoader takes up to 10 seconds, my CPU utilization drops significantly and dataloading becomes 3-4x slower.

I've tried including del vr and gc.collect() as from #1 (comment). I've also attempted to set the environment variable DECORD_NUM_THREADS as well different num_workers in the pytorch dataloader.