Unable to run Caldera 5.x on Ubuntu 20.04.6 LTS using either docker build or standard build

[robertstrom]

Describe the bug

I built a brand new virgin Ubuntu 20.04.6 VM that is fully updated and then walked through the instructions to install and run the latest version of Caldera.

When building the Docker instance I am unable to login using the documented credentials.

When attempting to launch Caldera non-Docker instance it fails to start

To Reproduce
Steps to reproduce the behavior:

• Docker attempted build

curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt install python3-pip

git clone https://github.com/mitre/caldera.git --recursive
cd caldera/
sudo apt  install docker.io
sudo usermod -aG docker $USER
newgrp docker
docker build . --build-arg WIN_BUILD=true -t caldera:latest

at this point I am able to get to a Caldera logon web page but none of the credentials are working for logon (credentials from the default.yaml file)

users:
  blue:
    blue: admin
  red:
    admin: admin
    red: admin

• Install / run non-Docker version

NOTE that dependencies are already installed as shown above.

• Launching with the following command

python3 server.py --insecure --build

Error text

Traceback (most recent call last):
  File "server.py", line 281, in <module>
    run_tasks(services=app_svc.get_services(), run_vue_server=args.uiDevHost)
  File "server.py", line 77, in run_tasks
    loop.run_until_complete(app_svc.register_contacts())
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/rstrom/caldera/app/service/app_svc.py", line 150, in register_contacts
    await self.register_contact_tunnels(contact_svc)
  File "/home/rstrom/caldera/app/service/app_svc.py", line 155, in register_contact_tunnels
    tunnel_class = import_module(tunnel_module_name).Tunnel
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/rstrom/caldera/app/contacts/tunnels/tunnel_ssh.py", line 3, in <module>
    import asyncssh
  File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/__init__.py", line 31, in <module>
    from .agent import SSHAgentClient, SSHAgentKeyPair, connect_agent
  File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/agent.py", line 34, in <module>
    from .public_key import KeyPairListArg, SSHCertificate, SSHKeyPair
  File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/public_key.py", line 35, in <module>
    from .crypto import ed25519_available, ed448_available
  File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/crypto/__init__.py", line 58, in <module>
    from .x509 import X509Certificate, X509Name, X509NamePattern
  File "/home/rstrom/.local/lib/python3.8/site-packages/asyncssh/crypto/x509.py", line 32, in <module>
    from OpenSSL import crypto
  File "/usr/lib/python3/dist-packages/OpenSSL/__init__.py", line 8, in <module>
    from OpenSSL import crypto, SSL
  File "/usr/lib/python3/dist-packages/OpenSSL/crypto.py", line 1553, in <module>
    class X509StoreFlags(object):
  File "/usr/lib/python3/dist-packages/OpenSSL/crypto.py", line 1571, in X509StoreFlags
    NOTIFY_POLICY = _lib.X509_V_FLAG_NOTIFY_POLICY
AttributeError: module 'lib' has no attribute 'X509_V_FLAG_NOTIFY_POLICY'

Expected behavior

• For the Docker build - Be able to log into Caldera
• For the standard build / execution - have Caldera start

Screenshots

Screenshots in-line above

Desktop (please complete the following information):

• OS - Ubuntu server

NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

• Browser - Chrome - Version 122.0.6261.129 (Official Build) (64-bit)

Additional context

• When running the Docker instance I am connecting to Caldera from a remote machine using the URL https://ipaddress:8888
• I also read another issues thread that updated npm using these commands and the build / execution still failed. Running the commands below put the npm version at v20.11.1

sudo npm cache clean -f
sudo npm install -g n
sudo n stable

[github-actions]

Looks like your first issue -- we aim to respond to issues as quickly as possible. In the meantime, check out our documentation here: http://caldera.readthedocs.io/

[robertstrom]

FYI - I built out another Ubuntu 20.04.6 LTS VM and recursively cloned the 4.2.0 branch and built the Docker container and have it running just fine. I am able to connect to it and login as I would expect to be able to do with version 5.

[TacitTactics]

I have tried to run on Ubuntu 23.10 the installation is successful and login appear but no activity when clicking the login button. I then tried to roll back to 4.2.0 the installation did not work at all for Ubuntu 22.04. It breaks at vite.js requiring a newer version of VUE. Manual installation of the VUE package does not affect the installation. I will try Ubuntu 20.04 with 4.2.0 and see what happens.

[TacitTactics]

I concur with @robertstrom, 4.2.0 on 20.04 works as expected. There is an incompatibility running 5.0.0 on any of the Ubuntu flavors, and an incompatibility between 4.2.0 and Ubuntu 22.04 +

[elegantmoose]

@TacitTactics Are you pulling 5.0.0 or Master. If the former, try pulling Master. We have putting patches in.

Also - this issue may apply here as well -> #2885

[TacitTactics]

Thank you for suggestion. I will try the master branch with both 22.04 and 23.10 and report back. To be clear, pulling from the master is leaving off the " --recursive" option, correct?

[elegantmoose]

Correct

[robertstrom]

cloned without the --recursive argument and I am getting this error

strom@caldera-5-ubuntu-2004:~/caldera$ python3 server.py --insecure --build
2024-03-25 20:29:58 WARNING  --insecure flag set. Caldera will use the default user accounts in default.yml config   server.py:216
                             file.                                                                                                
                    INFO     Using main config from conf/default.yml                                                 server.py:225
2024-03-25 20:29:59 INFO     Building VueJS front-end.                                                               server.py:261
npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: eslint-plugin-jinja2@0.1.0
npm ERR! Found: eslint@8.50.0
npm ERR! node_modules/eslint
npm ERR!   dev eslint@"^8.50.0" from the root project
npm ERR!   peer eslint@"^6.0.0 || ^7.0.0 || >=8.0.0" from @eslint-community/eslint-utils@4.4.0
npm ERR!   node_modules/@eslint-community/eslint-utils
npm ERR!     @eslint-community/eslint-utils@"^4.2.0" from eslint@8.50.0
npm ERR!   2 more (eslint-config-airbnb-base, eslint-plugin-import)
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer eslint@"^0.23.0 || ^1.0.0 || ^2.0.0" from eslint-plugin-jinja2@0.1.0
npm ERR! node_modules/eslint-plugin-jinja2
npm ERR!   dev eslint-plugin-jinja2@"^0.1.0" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: eslint@2.13.1
npm ERR! node_modules/eslint
npm ERR!   peer eslint@"^0.23.0 || ^1.0.0 || ^2.0.0" from eslint-plugin-jinja2@0.1.0
npm ERR!   node_modules/eslint-plugin-jinja2
npm ERR!     dev eslint-plugin-jinja2@"^0.1.0" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /home/rstrom/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/rstrom/.npm/_logs/2024-03-25T20_29_59_540Z-debug-0.log
Traceback (most recent call last):
  File "server.py", line 262, in <module>
    subprocess.run(["npm", "install"], cwd="plugins/magma", check=True)
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['npm', 'install']' returned non-zero exit status 1.

[robertstrom]

If you do not pull the repository recursively you get the error below when you try to build the Docker container

rstrom@caldera-5-ubuntu-20:~/caldera$ docker build . --build-arg WIN_BUILD=true -t caldera:latest
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  7.662MB
Step 1/44 : FROM ubuntu:23.04
23.04: Pulling from library/ubuntu
6360b3717211: Pull complete 
Digest: sha256:5a828e28de105c3d7821c4442f0f5d1c52dc16acf4999d5f31a3bc0f03f06edd
Status: Downloaded newer image for ubuntu:23.04
 ---> f4cdeba72b99
Step 2/44 : SHELL ["/bin/bash", "-c"]
 ---> Running in cbc00e3fa120
Removing intermediate container cbc00e3fa120
 ---> e93864fd5709
Step 3/44 : ARG TZ="UTC"
 ---> Running in deda115b86ff
Removing intermediate container deda115b86ff
 ---> 0e70b34ce51a
Step 4/44 : RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime &&     echo $TZ > /etc/timezone
 ---> Running in cf9afe446031
Removing intermediate container cf9afe446031
 ---> fb06543e92be
Step 5/44 : WORKDIR /usr/src/app
 ---> Running in 41e7e8031afc
Removing intermediate container 41e7e8031afc
 ---> 78101b512c12
Step 6/44 : ADD . .
 ---> 9f321a0604b3
Step 7/44 : RUN if [ -z "$(ls plugins/stockpile)" ]; then echo "stockpile plugin not downloaded - please ensure you recursively cloned the caldera git repository and try again."; exit 1; fi
 ---> Running in 10258785599f
stockpile plugin not downloaded - please ensure you recursively cloned the caldera git repository and try again.
The command '/bin/bash -c if [ -z "$(ls plugins/stockpile)" ]; then echo "stockpile plugin not downloaded - please ensure you recursively cloned the caldera git repository and try again."; exit 1; fi' returned a non-zero code: 1

[robertstrom]

I tried building the docker container after downloading the repository recursively. The build was successful but the same problem (not able to login) still exists

[Jaynornj]

I had the same issue with my 5.0.0

[elegantmoose]

@robertstrom You are pulling Master branch down recursively correct? (i.e. git clone https://github.com/mitre/caldera.git --recursive without specifying version. Do not specify v 5.0.0, you want Master as patches are being pushed there. When stable we will mint a version 5.1.0)

*FWIW, you cannot run Caldera without using the recursive flag, as some of the plugins have become a core dependency in v5, mainly the Magma plugin which is the entire new UI.

[elegantmoose]

@Jaynornj Same, try pulling Master, not version 5.0.0.

[rasmus1514]

@elegantmoose Also having the same problems even when pulling down the Master through "git clone https://github.com/mitre/caldera.git --recursive" does not seem to solve the issue sadly.

[dathtd119]

Ayo, I got this problem with V5.x on my Ubuntu 20.0.4 LTS. And you know what, I redo, but in server.py, i changed npm to pnpm, and boom, Solved!

I changed them at these