How handle parrallel jobs with long beets process import

[jee-r]

Hi,

My beets import process is very long :

First as usual beets search for candidates (musicbrainz, bandcamp etc...) and if a candidate match then :

• fetch metadata
• fetch art
• chroma
• replaygain
• copy in my media directory

If releases are imported one by one num_parallel_jobs = 1 there is no problem but this block all following imports if Betanin need a manual input as long as i resolve it so it's not the best way to import automatically.

if i set num_parallel_jobs = 4 for example, and if multiple import task are running in parallel i have the following issue :

[betanin] starting cli program

/torrents/seed/Music/Neil Young & Crazy Horse - Barn [Reprise 093624878438]  (2021) flac (10 items)

Tagging:
    Neil Young & Crazy Horse - Barn
URL:
    https://musicbrainz.org/release/67908d6d-8dc7-4eb2-98ca-a1b1b75c2e54
(Similarity: 99.7%) (tracks) (Digital Media, 2021, XW, Reprise Records)
 * Song Of The Seasons           -> Song of the Seasons
 * Change Ain't Never Gonna Come -> Change Ain’t Never Gonna (title)
 * Shape Of You                  -> Shape of You
 * Tumblin' Thru The Years       -> Tumblin’ Thru the Years
 * Don't Forget Love             -> Don’t Forget Love

Traceback (most recent call last):
  File "/usr/bin/beet", line 33, in 

    sys.exit(load_entry_point('beets==1.5.1', 'console_scripts', 'beet')())
  File "/usr/lib/python3.9/site-packages/beets/ui/__init__.py", line 1284, in main
    _raw_main(args)
  File "/usr/lib/python3.9/site-packages/beets/ui/__init__.py", line 1271, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/lib/python3.9/site-packages/beets/ui/commands.py", line 973, in import_func
    import_files(lib, paths, query)
  File "/usr/lib/python3.9/site-packages/beets/ui/commands.py", line 943, in import_files
    session.run()
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 340, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 446, in run_parallel
    raise exc_info[1].with_traceback(exc_info[2])
  File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 311, in run
    out = self.coro.send(msg)
  File "/usr/lib/python3.9/site-packages/beets/util/pipeline.py", line 170, in coro
    task = func(*(args + (task,)))
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 1444, in user_query
    apply_choice(session, task)
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 1515, in apply_choice
    task.add(session.lib)
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 773, in add
    self.record_replaced(lib)
  File "/usr/lib/python3.9/site-packages/beets/importer.py", line 788, in record_replaced
    dup_items = list(lib.items(
  File "/usr/lib/python3.9/site-packages/beets/library.py", line 1526, in items
    return self._fetch(Item, query, sort or self.get_default_item_sort())
  File "/usr/lib/python3.9/site-packages/beets/library.py", line 1500, in _fetch
    return super()._fetch(
  File "/usr/lib/python3.9/site-packages/beets/dbcore/db.py", line 1094, in _fetch
    flex_rows = tx.query(flex_sql, subvals)
  File "/usr/lib/python3.9/site-packages/beets/dbcore/db.py", line 858, in query
    cursor = self.db._connection().execute(statement, subvals)
sqlite3.OperationalError: database is locked

[betanin] program finished with exit status `0`

The first import task locked the beets database and all others imports will failed with exit status 0

do you know how i can solve this issue ? it's not easy to explain this issue i hope it's understandable enough

--
Ps: thank you very much for betanin it's an awesome tool

[sentriz]

this one is hard to reproduce for me, haven't seen the database is locked error with num_parallel_jobs > 1