sqlite3.DatabaseError: database disk image is malformed

[mihalycsaba]

What's the bug all about?

I ran the pipupgrade command once it looked like it worked, second time it doesnt work.

What command led to this issue?

$ "<pipupgrade --pip-path /usr/bin/pip>"

Logs

Please paste the logs using pipupgrade --verbose

"<pipupgrade --pip-path /usr/bin/pip --verbose
2021-05-05 14:07:09,243 | INFO | Environment: {'version': '1.9.0', 'python_version': '3.6.8', 'os': 'Linux-3.10.0-1160.24.1.el7.x86_64-x86_64-with-centos-7.9.2009-Core', 'config': {'path': {'BASE': '/usr/local/lib/python3.6/site-packages/pipupgrade', 'DATA': '/usr/local/lib/python3.6/site-packages/pipupgrade/data', 'CACHE': '/root/.config/pipupgrade', 'JOBS': '/usr/local/lib/python3.6/site-packages/pipupgrade/jobs'}}, 'settings': {'settings': {'version': '1.9.0', 'cache_timeout': '86400', 'jobs': '4'}}}
2021-05-05 14:07:09,244 | INFO | Arguments Passed: {'a': <pipupgrade.to_params.<locals>.O object at 0x7f839cad8748>, 'kwargs': {'ARGUMENTS': <class 'inspect._empty'>, 'packages': [], 'ignore': None, 'pip_path': ['/usr/bin/pip'], 'yes': 0, 'check': 0, 'upgrade_type': ['minor', 'patch'], 'latest': 0, 'format': 'table', 'all': 0, 'pip': 0, 'self': False, 'requirements': None, 'pipfile': None, 'interactive': 0, 'project': None, 'git_username': None, 'git_email': None, 'pull_request': False, 'github_access_token': None, 'github_reponame': None, 'github_username': None, 'target_branch': 'master', 'jobs': 4, 'user': 0, 'no_included_requirements': 0, 'no_cache': 0, 'output': None, 'ignore_error': 0, 'force': 0, 'no_color': 0, 'verbose': True}, 'args': ()}
Checking...
2021-05-05 14:07:09,247 | INFO | `pip` executables found: [None]
2021-05-05 14:07:09,247 | INFO | Using 4 jobs...
2021-05-05 14:07:09,267 | INFO | Fetching installed packages for None...
2021-05-05 14:07:09,269 | INFO | Executing command: /usr/local/bin/pip list --outdated --format json
2021-05-05 14:07:15,761 | INFO | 8 packages found for None.
2021-05-05 14:07:15,779 | INFO | Initializing Package {'name': 'aenum', 'version': '2.2.3', 'latest_version': '3.0.0', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,779 | INFO | Initializing Package {'name': 'certifi', 'version': '2019.11.28', 'latest_version': '2020.12.5', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,780 | INFO | Fetching package aenum information from DB...
2021-05-05 14:07:15,779 | INFO | Initializing Package {'name': 'chardet', 'version': '3.0.4', 'latest_version': '4.0.0', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,780 | INFO | Fetching package certifi information from DB...
2021-05-05 14:07:15,780 | INFO | Fetching package chardet information from DB...
2021-05-05 14:07:15,780 | INFO | Initializing Package {'name': 'cssutils', 'version': '1.0.2', 'latest_version': '2.2.0', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,780 | INFO | Fetching package cssutils information from DB...
2021-05-05 14:07:15,784 | INFO | Initializing Package {'name': 'idna', 'version': '2.10', 'latest_version': '3.1', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,784 | INFO | Fetching package idna information from DB...
2021-05-05 14:07:15,784 | INFO | Initializing Package {'name': 'Pillow', 'version': '7.0.0', 'latest_version': '8.2.0', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,784 | INFO | Fetching package Pillow information from DB...
2021-05-05 14:07:15,785 | INFO | Initializing Package {'name': 'setuptools', 'version': '39.2.0', 'latest_version': '56.1.0', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,785 | INFO | Fetching package setuptools information from DB...
2021-05-05 14:07:15,785 | INFO | Initializing Package {'name': 'xlrd', 'version': '1.2.0', 'latest_version': '2.0.1', 'latest_filetype': 'wheel'} of type <class 'dict'>...
2021-05-05 14:07:15,785 | INFO | Fetching package xlrd information from DB...
2021-05-05 14:07:15,790 | INFO | Using cached info for package <Package certifi (2019.11.28)>.

multiprocessing.pool.RemoteTraceback:
"""
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.6/site-packages/pipupgrade/model/package.py", line 115, in __init__
    """ % self.name)
  File "/usr/local/lib/python3.6/site-packages/pipupgrade/db.py", line 53, in query
    )(*args, **kwargs)
sqlite3.DatabaseError: database disk image is malformed
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/local/lib/python3.6/site-packages/pipupgrade/commands/helper.py", line 335, in get_registry_from_pip
    resolve = resolve, jobs = jobs, latest = latest)
  File "/usr/local/lib/python3.6/site-packages/pipupgrade/model/registry.py", line 117, in __init__
    for package in pool.imap_unordered(partial(Package, **args), packages):
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 735, in next
    raise value
sqlite3.DatabaseError: database disk image is malformed
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pipupgrade/commands/__init__.py", line 78, in command
    return _command(**ARGUMENTS)
  File "/usr/local/lib/python3.6/site-packages/pipupgrade/commands/__init__.py", line 233, in _command
    pip_path
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 735, in next
    raise value
sqlite3.DatabaseError: database disk image is malformed

2021-05-05 14:07:15,793 | INFO | Using cached info for package <Package cssutils (1.0.2)>.
2021-05-05 14:07:15,793 | INFO | Using cached info for package <Package Pillow (7.0.0)>.
An error occured while performing the above command. This could be an issue with
"pipupgrade". Kindly post an issue at https://github.com/achillesrasquinha/pipupgrade/issues
>"

[Janosch]

I got the same thing. In my case I suspect it has something to do with a previously failed upgrade attempt:

Updating 24 of 35 packages: pycurl

Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/commands/__init__.py", line 78, in command
    return _command(**ARGUMENTS)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/commands/__init__.py", line 244, in _command
    upgrade_type = a.upgrade_type
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/commands/helper.py", line 305, in update_registry
    upgrade = True
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/_pip.py", line 75, in call
    output = popen(*params, output = output, raise_err = raise_err)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/util/system.py", line 92, in popen
    raise PopenError(code, command)
pipupgrade.exception.PopenError: Command '/usr/bin/pip install pycurl --no-cache-dir --upgrade' returned non-zero exit status 1.

When I try to do pipupgrade again, I get the same error as @mihalycsaba:

janosch@icomplai.com:~/environment $ pipupgrade
Checking...

multiprocessing.pool.RemoteTraceback: 
"""
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/model/package.py", line 115, in __init__
    """ % self.name)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/db.py", line 53, in query
    )(*args, **kwargs)
sqlite3.DatabaseError: database disk image is malformed
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/commands/helper.py", line 335, in get_registry_from_pip
    resolve = resolve, jobs = jobs, latest = latest)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/model/registry.py", line 117, in __init__
    for package in pool.imap_unordered(partial(Package, **args), packages):
  File "/usr/lib64/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
sqlite3.DatabaseError: database disk image is malformed
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/commands/__init__.py", line 78, in command
    return _command(**ARGUMENTS)
  File "/home/ec2-user/.local/lib/python3.7/site-packages/pipupgrade/commands/__init__.py", line 233, in _command
    pip_path
  File "/usr/lib64/python3.7/multiprocessing/pool.py", line 748, in next
    raise value
sqlite3.DatabaseError: database disk image is malformed

[achillesrasquinha]

I'll be introducing pipupgrade --doctor soon to rectify this issue. For now, you could clean the cache by simply rm -rf ~/.config/pipupgrade and try again

[mrichman]

This happens every time I run the command after deleting ~/.config/pipupgrade.

[mrichman]

I will need to stop using this tool until this issue is resolved. It's quite annoying. Deleting ~/.config/pipupgrade doesn't solve the problem for me.