Release 2024.1.0

README.containers.md

@@ -8,14 +8,14 @@ There are a number of containers that make up the application. Here are quick de
 3. [Binary Analysis Worker Container (bdba-worker)](#-binary-analysis-worker-container-bdba-worker)
 4. [CA Container (blackduck-cfssl)](#-ca--container-blackduck-cfssl)
 5. [Documentation Container (blackduck-documentation)](#-documentation-container-blackduck-documentation)
-6. [Integration Container (blackduck-integration)])(#-integration-container-artifactory-integration)
+6. [Integration Container (blackduck-integration)](#-integration-container-artifactory-integration)
 7. [Job Runner Container (blackduck-jobrunner)](#-job-runner-container-blackduck-jobrunner)
 7. [LogStash Container (blackduck-logstash)](#-logstash--container-blackduck-logstash)
 8. [Match Engine Container (blackduck-matchengine)](#-matchengine-container-blackduck-matchengine)
 9. [RabbitMQ Container (rabbitmq)](#-rabbitmq-container-rabbitmq)
 10. [Registration Container (blackduck-registration)](#-registration-container-blackduck-registration)
 11. [Scan Container (blackduck-scan)](#-scan-container-blackduck-scan)
-12. [Upload Cache Container (blackduck-upload-cache)](#-upload-cache-container-blackduck-upload-cache)
+12. [Storage Container (blackduck-storage)](#-storage-container-blackduck-storage)
 13. [Web App Container (blackduck-webapp)](#-web-app-container-blackduck-webapp)
 14. [Web Server Container (blackduck-nginx)](#-web-server-container-blackduck-nginx)
 
@@ -205,11 +205,15 @@ This container is also able to be started as a random UID as long as it is also
 
 ## Container Description
 
-The object storage service stores tools (files) for use by Detect.
+The object storage service stores tools (files) for use by Detect,
+generated reports, uploaded SBOMs, BDIO files, and other bulk data.
+If the Black Duck Binary Analysis feature is enabled uploaded binary
+files are stored here temporarily. If the source view feature is
+enabled source files are stored here.
 
 ## Scalability
 
-This container can be scaled, but if using a File storage provider all instance must share the same persistent volume.
+This container can be scaled, but if using a File storage provider all replicas must share the same persistent volume.
 
 ## Links/Ports
 
@@ -232,6 +236,11 @@ Compose or Docker Swarm use. These environment variables can be set to override
 * cfssl - $HUB_CFSSL_HOST
 * rabbitmq - $RABBIT_MQ_HOST
 
+## Other configurable environment variables
+
+* Default disk size for source files: 4GB ($MAX_TOTAL_SOURCE_SIZE_MB)
+* Default Data Retention Days: 180 ($DATA_RETENTION_IN_DAYS)
+
 ## Users/Groups
 
 This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing its main process.
@@ -451,7 +460,7 @@ This container will need to connect to these other containers/services:
 * documentation
 * scan
 * authentication
-* upload cache
+* storage
 
 This container should expose port 443 outside of the docker network.
 
@@ -467,7 +476,7 @@ There are times when running in other types of orchestrations that any individua
 * matchengine - $HUB_MATCHENGINE_HOST
 * cfssl - $HUB_CFSSL_HOST
 * documentation - $HUB_DOC_HOST
-* upload cache - $HUB_UPLOAD_CACHE_HOST
+* storage - $BLACKDUCK_STORAGE_HOST
 
 ## Users/Groups
 
@@ -575,54 +584,6 @@ This container runs as UID 100. If the container is started as UID 0 (root) then
 This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).
 
 
-# Upload Cache Container (blackduck-upload-cache)
-----
-
-## Container Description
-
-This container mainly stores customer's files that are consumed by the other services. Currently, the main purpose of the service is to store the source files
-for the source
-view feature (starting 2019.04 release) and temporarily storing the binary files for the binary analysis (if Black Duck Binary Analysis feature is enabled.)
-
-## Scalability
-
-There should only be a single instance of this container. It currently cannot be scaled.
-
-## Links/Ports
-
-This container will need to connect to these other containers/services:
-
-* cfssl
-* logstash
-
-And if Black Duck Binary Analysis is enabled:
-
-* rabbitmq
-
-The container will need to expose ports 9443 and 9444 to other containers that will link to it.
-
-## Alternate Host Name Environment Variables
-
-There are times when running in other types of orchestrations that any individual service name may be different. For example, You may have an external logstash
-endpoint which is resolved through a different service name.
-
-To support any such use case, these environment variables can be set to override the default service names:
-
-* cfssl - $HUB_CFSSL_HOST
-* logstash - $HUB_LOGSTASH_HOST
-* rabbitmq - $RABBIT_MQ_HOST
-
-## Other configurable environment variables
-
-* Default disk size for source files: 4GB ($MAX_TOTAL_SOURCE_SIZE_MB)
-* Default Data Retention Days: 180 ($DATA_RETENTION_IN_DAYS)
-
-## Users/Groups
-
-This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing its main process.
-This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).
-
-
 # Binary Analysis Worker Container (bdba-worker)
 ----
 
@@ -688,4 +649,4 @@ This container will need to expose port 8443 to other containers that will link
 ## Users/Groups
 
 This container runs as UID 100. If the container is started as UID 0 (root) then the user will be switched to UID 100:root before executing its main process.
-This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).
+This container is also able to be started as a random UID as long as it is also started within the root group (GID/fsGroup 0).

README.md

@@ -2,9 +2,9 @@
 
 This repository contains orchestration files and documentation for deploying Black Duck Docker containers.
 
-## Location of Black Duck 2023.10.2 archive:
+## Location of Black Duck 2024.1.0 archive:
 
-https://github.com/blackducksoftware/hub/archive/v2023.10.2.tar.gz
+https://github.com/blackducksoftware/hub/archive/v2024.1.0.tar.gz
 
 NOTE:
 
@@ -39,7 +39,6 @@ https://github.com/blackducksoftware/hub/releases
 * https://hub.docker.com/r/blackducksoftware/blackduck-scan/
 * https://hub.docker.com/r/blackducksoftware/blackduck-storage/
 * https://hub.docker.com/r/blackducksoftware/blackduck-webapp/
-* https://hub.docker.com/r/blackducksoftware/blackduck-upload-cache/
 * https://hub.docker.com/r/blackducksoftware/blackduck-redis/
 * https://hub.docker.com/r/blackducksoftware/blackduck-matchengine/
 * https://hub.docker.com/r/sigsynopsys/bdba-worker/

docker-swarm/README.md

@@ -531,44 +531,8 @@ secrets:
 
 # Source Upload Feature 
 
-Source side by side view feature is included in 2019.04 release. In order to enable the feature, there are two steps need to be done before the deployment.
+Source side by side view feature is included in 2019.04 release. In order to enable the feature you need to enable it in 'blackduck-config.env':
 
-**1. The flag in blackduck-config.env should be set to true.** 
 ```
 ENABLE_SOURCE_UPLOADS=true
 ```
-**2. Seal Key creation.**
-
-When source files are uploaded, they are stored encrypted in the container (upload cache service). 
-
-Black Duck requires customers to provide their own seal key which is 32 bytes long in order to support the AES-256 encryption. And the seal key needs to be provided to the upload cache service. 
-
-Under the uploadcache service configuration in docker-compose.yml, provide the location where you keep the file.
-
-```
-uploadcache:
-    secrets:
-      - SEAL_KEY
-```
-And define the top level secrets at the bottom of the docker-compose.yml file as:
-```
-secrets:
-  SEAL_KEY:
-   external: true
-   name: "hub_SEAL_KEY"
-```
-
-**NOTE: If the seal key isn't provided, the source side by side view feature won't be available in Black Duck**
-
-
-### Key recovery support
-
-The upload cache service encrypts the file data with a root key. The root key is generated at the very first start of the application.
-The key can only be retrieved with the seal key, thus the encrypted data cannot be decrypted when the seal key isn't available.
-
-To protect the loss of file data, Black Duck supports the key recovery on demand. If customer wishes to retrieve the root key, they can do so by running the script as below.
-The script requires two arguments, local destination where you wish to store the root key (**please make sure to place it in a secure location**) and a path where you keep the seal key.
-
-```
-./bin/bd_get_source_upload_master_key.sh <local_destination_directory_path> <seal_key_file_path>
-```

docker-swarm/bin/hub_add_replication_user.sh

@@ -3,7 +3,7 @@
 set -e
 
 TIMEOUT=${TIMEOUT:-10}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.17}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.20}
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
 
 function fail() {

docker-swarm/bin/hub_create_data_dump.sh

@@ -5,8 +5,8 @@
 #  2. The database container has been properly initialized.
 
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.17}
-HUB_VERSION=${HUB_VERSION:-2023.10.2}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.20}
+HUB_VERSION=${HUB_VERSION:-2024.1.0}
 OPT_FORCE=
 OPT_LIVE_SYSTEM=
 OPT_MAX_CPU=${MAX_CPU:-1}

docker-swarm/bin/hub_db_migrate.sh

@@ -14,7 +14,7 @@
 set -o errexit
 
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.17}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.20}
 OPT_MAX_CPU=${MAX_CPU:-1}
 OPT_NO_DATABASE=${NO_DATABASE:-}
 OPT_NO_STORAGE=${NO_STORAGE:-}

docker-swarm/bin/hub_replication_changepassword.sh

@@ -3,7 +3,7 @@
 set -e
 
 TIMEOUT=${TIMEOUT:-10}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.17}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.20}
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
 
 function fail() {

docker-swarm/bin/hub_reportdb_changepassword.sh

@@ -3,7 +3,7 @@
 set -e
 
 TIMEOUT=${TIMEOUT:-10}
-HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.17}
+HUB_POSTGRES_VERSION=${HUB_POSTGRES_VERSION:-14-1.20}
 HUB_DATABASE_IMAGE_NAME=${HUB_DATABASE_IMAGE_NAME:-postgres}
 
 function fail() {