Docker-Compose Upgrade from 6.17 to 7.3.2 not working for docker-volumes

[kb7791]

Current:
postgres:9.6.11
artifactory-pro:6.17.0

Upgrade:
postgres:9.6.11
artifactory-pro:7.3.2

The current installation is running from a docer-compose file, botht the artifactory 6.17 and postgres 9.6.11 are volume mounted to docker named volumes and not bind mounts,

The directions to upgrade from 6.17 to 7.3.2 (https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-DockerComposeUpgrade) I find to be a little vague.

1. Stop and rm my artifactory and postgresql containers.
2. I have the tarball extracted for 7.3.2.
3. Run config.sh

Not really sure what this is doing but it looks like to me that its creating a whole new directory to install all the 7.3.2 data and content in /root/.jfrog/artifactory, wouldn't that cause me to not mount my existing artifactory_data and posgtres_data docker volumes to this upgrade procedure and just create a new empty artifactory 7.3.2. I modified the docker-compose-postgres.yaml to use my existing docker volumes (artifactory_data and postgres_data that reside in /var/lib/docker/volumes but when artifactory 7.3.2 came up its was prompting me for license again and the data and existing user were invalid. If I change the ROOT_DATA_DIR in .env file it looks as if redundant directories are created with in a var directory the script creates that should be holding the artifactory data.

Not sure if I'm missing something here on of these steps. Not sure if something needs to be changed in config.sh file or one of the other underlying helper files to get my previous volumes to work. Kind of weary of changing anything in fear of breaking the upgrade progress that jfrog implemented.

[JfrogPrasanna]

@kb7791 First of all this repo is deprecated. Please use the latest version from wiki.

if you look at the volume set up in the new version, it is done as below:

${ROOT_DATA_DIR}/var:/var/opt/jfrog/artifactory

Where the actual mount point is ROOT_DATA_DIR/var. The reason is we have standardixzed this across all installers to make the structure to have JFROG_HOME/artifactory/var as the data directory and JFROG_HOME/artifactory/app as the binary directory (which changes every release)

if your data is not under var, you can change this (${ROOT_DATA_DIR}/var) to not include var. The ./config.sh only sets up the directory, permissions and add system.yaml information. If you are setting them up manually then there is no need to run config.sh

[kb7791]

@JfrogPrasanna I understand that this is deprecated, but I'm trying to migrate from your now deprecated docker-compose artifactory-postgresql example for 6.x to 7.3.2 and this link is still being used for tracking issues on your official official Docker-Compose Upgrade instructions to 7.3.2.

So if I have a 2 separate named docker volumes for artifactory_data and postgresql_data, shouldn't I be able to avoid using the conifig.sh process as a whole and go into the templates directory within the new tarball for 7.3.2 and modify the docker-compose-postgres.yaml that looks like the following...

version: '3'
services:
  postgres:
    image: ${DOCKER_REGISTRY}/postgres:9.6.11
    container_name: postgresql
    environment:
     - POSTGRES_DB=artifactory
     - POSTGRES_USER=artifactory
     - POSTGRES_PASSWORD=password
    ports:
      - 5432:5432
    volumes:
     - ${ROOT_DATA_DIR}/var/data/postgres/data:/var/lib/postgresql/data
     - /etc/localtime:/etc/localtime:ro
    restart: always
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
  artifactory:
    image: ${DOCKER_REGISTRY}/jfrog/artifactory-pro:${ARTIFACTORY_VERSION}
    container_name: artifactory
    volumes:
     - ${ROOT_DATA_DIR}/var:/var/opt/jfrog/artifactory
     - /etc/localtime:/etc/localtime:ro
    restart: always
    depends_on:
    - postgres
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
    environment:
     - JF_ROUTER_ENTRYPOINTS_EXTERNALPORT=${JF_ROUTER_ENTRYPOINTS_EXTERNALPORT}
    ports:
      - ${JF_ROUTER_ENTRYPOINTS_EXTERNALPORT}:${JF_ROUTER_ENTRYPOINTS_EXTERNALPORT} # for router communication
      - 8081:8081 # for artifactory communication
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"

The upgrade procedure states that I should be able to stop my current docker-compose containers and remove them. If my artifactory data and postgres data already exists on my local filesystem that is managed by Docker and already ahs the permissions setup so that artifactory 6.17.0 can read and write from without issues. Should I be able to modify the template that in the 7.3.2 package name docekr-compose-postgres.yaml to look like the following...

version: '3'
services:
  postgres:
    image: docker.bintray.io/postgres:9.6.11
    container_name: postgresql
    environment:
     - POSTGRES_DB=artifactory
     - POSTGRES_USER=xxxxx
     - POSTGRES_PASSWORD=xxxxx
    ports:
      - 5432:5432
    volumes:
     - postgresql_data:/var/lib/postgresql/data
     - /etc/localtime:/etc/localtime:ro
    restart: always
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
  artifactory:
    image: docker.bintray.io/jfrog/artifactory-pro:7.3.2
    container_name: artifactory
    volumes:
     - artifactory_data:/var/opt/jfrog/artifactory
     - /etc/localtime:/etc/localtime:ro
    restart: always
    depends_on:
    - postgres
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
    ports:
      - 8081:8081 # for artifactory communication
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"
volumes:
    postgres_data:
        external: true
    artifactory_data:
        external: true

and docker-compose back up and all of the existing data in the mounts that were working with 6.17.0 be working with the 7.3.2 version? Based on the instructions on
https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-DockerComposeUpgrade

Has the directory structure changed from 6.17.0 that...
is no longer valid? Unless there are a bunch of manual steps of moving things out of old directories in the mount to a new locations that I need to take to get this working correctly.

Your upgrade steps on the link mentioned above literally have 7 steps and if there are underlying changes that need to be made to go from from a docker-compose 6.17 to docker-compose 7.2.3 upgrade, these 7 steps are a little vague and lacking direction.

[kb7791]

              _   _  __           _                     _____
     /\        | | (_)/ _|         | |                   |  __ \
    /  \   _ __| |_ _| |_ __ _  ___| |_ ___  _ __ _   _  | |__) | __ ___
   / /\ \ | '__| __| |  _/ _` |/ __| __/ _ \| '__| | | | |  ___/ '__/ _ \
  / ____ \| |  | |_| | || (_| | (__| || (_) | |  | |_| | | |   | | | (_) |
 /_/    \_\_|   \__|_|_| \__,_|\___|\__\___/|_|   \__, | |_|   |_|  \___/
 Version:  7.3.2                                   __/ |
 Revision: 70302900                               |___/
 Artifactory Home: '/opt/jfrog/artifactory'
 Node ID: '9cef3e14aca6'

2020-04-13T22:43:45.384L [tomct] [INFO ] [                ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deployment of deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml] has finished in [4,353] ms
2020-04-13T22:43:45.455Z [jfrt ] [INFO ] [4fd32a8104278226] [ifactoryApplicationContext:523] [art-init            ] - Artifactory application context set to NOT READY by refresh
2020-04-13T22:43:46.659Z [jfac ] [INFO ] [68f28b9129675d23] [s.d.u.AccessJdbcHelperImpl:138] [ocalhost-startStop-1] - Database: Apache Derby 10.14.2.0 - (1828579). Driver: Apache Derby Embedded JDBC Driver 10.14.2.0 - (1828579)
2020-04-13T22:43:46.660Z [jfac ] [INFO ] [68f28b9129675d23] [s.d.u.AccessJdbcHelperImpl:141] [ocalhost-startStop-1] - Connection URL: jdbc:derby:{db.home}
2020-04-13T22:43:46.670Z [jfac ] [INFO ] [68f28b9129675d23] [s.d.u.AccessJdbcHelperImpl:150] [ocalhost-startStop-1] - ***Creating database schema***
2020-04-13T22:43:46.959Z [jfac ] [ERROR] [68f28b9129675d23] [o.j.s.u.DbStatementUtils:66   ] [ocalhost-startStop-1] - Failed to execute query: TYPE 'BYTEA' does not exist.:
CREATE TABLE access_permissions_custom_data (
  perm_id    BIGINT   NOT NULL,
  data       BYTEA    NOT NULL,
  CONSTRAINT access_perm_data_pk PRIMARY KEY (perm_id),
  CONSTRAINT access_perm_data_perms_fk FOREIGN KEY (perm_id) REFERENCES access_permissions (perm_id)
)

this is with postgresql not only set in the system yaml but also setup in the docker compose file, seems as if derby still want to start up.

[JfrogPrasanna]

@kb7791 We were able to simulate this and here is the root cause.

1. The folder structure of data has in-fact changed significantly between 6.x and 7.x
2. The config.sh runs on the host and tries migrated to the new folder structure and also migrate the properties to the new system.yaml. But it fails for docker volumes of the kind above due to permission issues.
3. To handle this, you may want to bring a empty container (like a busy box and perform the migration manually as explained in here or perform a upgrade of folder structure via a simple docker run (without compose) as explained here (this is available only from 7.4.1)
4. Once migration is complete, you can run the compose and it should be fine.

Let me know if this works.

[JakeDEvans]

@JfrogPrasanna you need to throw out your docker-compose 'config.sh' system and start over.

The docker image itself should check the version of the data and do the migration if needed.

1 volume for data
1 volume for db (if external)

Do not publicly maps all ports, only map ports you need (expose in Dockerfile, link w/compose, publicly expose nginx)

HA should be managed with a route/api endpoint, not a specialty port or by exposing the app port on plaintext/non-ssl ports.

looking forward to 7.5.9

[kb7791]

@JfrogPrasanna
The issue I'm having here and someone correct me if I'm wrong. Is that your config file is essentially taking a docker volume that is managed by docker and going in having to chown it to 1030:1030 essentially turning it back into a local mountthat is no longer manged by Docker and not letting Docker do its job.

The docker-compose and system.yaml to use posgresql were completely ignored during one of my last attempts and still booted up a derby.db as you can see from the Artifactory stack trace, wasn't sure what happened there.

We're upgrading to the latest 6.19.0 for now and will revisit upgrading to 7.x in the near future once this migration process is simplified or complexity is ironed out.

[JfrogPrasanna]

@JakeDEvans @kb7791 Thanks for your feedback - We will document the steps to handle docker volumes, which is clearly missed in our documentation and scripts.

[kad-meedel]

@kb7791 We were able to simulate this and here is the root cause.

1. The folder structure of data has in-fact changed significantly between 6.x and 7.x
2. The config.sh runs on the host and tries migrated to the new folder structure and also migrate the properties to the new system.yaml. But it fails for docker volumes of the kind above due to permission issues.
3. To handle this, you may want to bring a empty container (like a busy box and perform the migration manually as explained in here or perform a upgrade of folder structure via a simple docker run (without compose) as explained here (this is available only from 7.4.1)
4. Once migration is complete, you can run the compose and it should be fine.

Let me know if this works.

Manually upgrade is not possible. We have 3 TB of filestore.
I have the same issue. The installation log mention the postgresql settings, but still derby is used for the database!?

What I need is the installation document for a fresh installation of 7.4.x with docker compose and the combination of artifactory with nginx and postgresql.
Is there any document or example for this combination and a new fresh installation?

[JfrogPrasanna]

@kad-meedel The issue here in this issue is that of a compose file with docker-volume. Are you using docker-volume as well? If you are, we are working on a new migration procedure to address it coming in the upcoming release. If not, please send us the system.yaml to validate what it is updated with.

[kad-meedel]

@JfrogPrasanna Thanks for your reply. Have you any timeschedule when this new procudure / new release wil be available?

The system.yaml within the artifactory docker container has not been updated with the settings from the docker-compose.yaml input file.
There are only example rules.

---

system.yaml
!## Example for postgresql

#type: postgresql

#driver: org.postgresql.Driver

#url: jdbc:postgresql://<your db url, for example: localhost:5432>/artifactory

#username: artifactory

#password: password

[JfrogPrasanna]

system.yaml will not be updated from docker-compose ENV vars. ENV vars override system values and their names should be of type JF_SHARED_DATABASE_TYPE to represent system.yaml entry

shared: 
  database: 
    type:

The timeline we are looking for the support for docker-volumes is sometime May. I would not know the exact release date yet.

[deeco]

What I need is the installation document for a fresh installation of 7.4.x with docker compose and the combination of artifactory with nginx and postgresql.
Is there any document or example for this combination and a new fresh installation?

Did you ever find this ?

[eldada]

@deeco - Look at the official documentation.

[JakeDEvans]

Something like this... both of my images have some customization for internal certs and plugins.

version: '3'
services:
  postgresql:
    image: postgres:9.6.11
    environment:
     - POSTGRES_DB=artifactory
     - POSTGRES_USER=artifactory
     - POSTGRES_PASSWORD=$ARTIFACTORY_POSTGRES_PASSWORD
    volumes:
     - postgres_data:/var/lib/postgresql/data
    restart: always
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
  artifactory:
    image: artifactory_with_custom_ca:7.7.3
    depends_on:
     - postgresql
    volumes:
     - artifactory_data:/var/opt/jfrog/artifactory
    environment:
     - EXTRA_JAVA_OPTIONS=-Djavax.net.ssl.trustStore=/var/opt/jfrog/artifactory/etc/cacerts -Djavax.net.ssl.trustStorePassword=changeit
    restart: always
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
  nginx: nginx_with_custom_ssl
    image: 
    ports:
     - 80:80
     - 443:443
    depends_on:
     - artifactory
    restart: always
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000
volumes:
  postgres_data:
    driver_opts:
      type: none
      device: /srv/artifactory/postgres_data
      o: bind
  artifactory_data:
    driver_opts:
      type: none
      device: /srv/artifactory/artifactory_data
      o: bind

you have to seed the postgres password, they can't seem to figure out how docker uses env vars.

mkdir -p /srv/artifactory/artifactory_data/etc/ /srv/artifactory/postgres_data/
cat << EOF > /srv/artifactory/artifactory_data/etc/system.yaml
shared:
    node:
        ip: artifactory
        id: artifactory
        name: artifactory
        contextUrl: http://artifactory:8082/artifactory
    database:
        type: postgresql
        driver: org.postgresql.Driver
        username: artifactory
        password: art_password
        url: jdbc:postgresql://postgresql:5432/artifactory
router:
    entrypoints:
        externalPort: 8082

EOF
chown 1030:1030 -R /srv/artifactory/artifactory_data/

[deeco]

@JakeDEvans this helps thank you , I was using similar to https://github.com/deeco/artifactory-install/tree/master/artifactory/artifactory which works fine for version 6 just not version 7

[JfrogPrasanna]

Did the official documentation which now supports docker volume not help? Where volumes are pre-created?

Here is the updated docs for upgrade and new installation:
New: https://www.jfrog.com/confluence/display/JFROG/Installing+Artifactory#InstallingArtifactory-DockerInstallation
Upgrade: https://www.jfrog.com/confluence/display/JFROG/Upgrading+Artifactory#UpgradingArtifactory-DockerUpgradedockerupgr6

Hope it helps.

[kb7791]

I haven't had to time to re-attempt the upgrade. We ended having to migrate over to bind mount versus docker volume and are still on the latest 6.x. I'll be re-attempting a move to 7.x in the near future once some time is allotted for the upgrade.