Extreme slow rewriting of files on NFS drive

[crankedguy]

Hello,

I noticed that there are extremely slow speeds when files have to be rewritten. The drive is an NFS mount and a 200MB FLAC needs over 30 seconds to be rewritten. Network transfer speeds are capped at 30MB/s shown during the operation.
Normal speeds are shown around 125/110 read/write
Any idea why this is the case with Mutagen, I have to rewrite several TB and this cannot be done so.

Thanks

[lazka]

Likely because the rewrite is done in blocks:

mutagen/mutagen/_util.py

Line 722 in 399513b

def move_bytes(fobj, dest, src, count, BUFFER_SIZE=_DEFAULT_BUFFER_SIZE):

You could try increasing _DEFAULT_BUFFER_SIZE (in the code)

[crankedguy]

Thanks for your quick response. Any idea how to circumvent this except copying the files to tag to a local disk and back? Which is painful in its own with these sizes

[crankedguy]

I've seen you just answered this :) Will try it out giving it some more buffer, thanks

[lazka]

Otherwise you can use the following from https://mutagen.readthedocs.io/en/latest/user/padding.html

def no_new_padding(info):
    # this will use existing padding but never add new one
    return max(info.padding, 0)

f.save(padding=no_new_padding)

This means it will only rewrite the file if absolutely needed. By default it will add more padding the first time to make future changes are faster.

If the files don't have any tags at all, or no padding, or you add cover art, then this wont help though.

[crankedguy]

Yes this is the case, most of the files have no cover art at all and if, it has to be updated, also they are not padded well enough by the creator. I will never understand why someone doesn't pad a few bytes nowadays when creating a 2-300MB in the first place. But it is how it is

[crankedguy]

I notched this up to 2**21 until now and it brought tremendous speed improvements. Still slow though in terms to normal speed of the shares.

[phw]

There is also related #587