Externalizing environment-specific microservice configuration for CI/CD

Note	This repository contains the guide documentation source. To view the guide in published form, view it on the Open Liberty website.

Learn how to create environment-specific configurations for microservices by using MicroProfile Config’s configuration profile feature for smoother management throughout the CI/CD lifecycle.

What you’ll learn

In the domain of software delivery, managing configurations for microservices can become complex, especially when configurations require adjustments across various stages of the software lifecycle. MicroProfile Config’s configuration profile feature, also known as the Config Profile, is a direct solution to this challenge. It simplifies the management of microservice configurations across diverse environments - from development to production and throughout the CI/CD pipeline. By tailoring configuration properties to each environment, the deployment process becomes more seamless, allowing developers to concentrate on perfecting their application’s delivery.

You’ll learn how to provide environment-specific configurations by using the MicroProfile Config’s configuration profile feature. You’ll create configuration profiles at both the individual property level and the higher-level configuration sources.

This guide builds on the basics of the Separating configuration from code in microservices guide and the Configuring microservices guide. If you are not familiar with externalizing the configuration of microservices, it will be helpful to read the External configuration of microservices document and complete those guides before proceeding.

The application that you will work with is a query service, which fetches information about the running JVM from a system microservice. Given the variances in accessing parameters between development, testing, and production environments, you’ll use the configuration profile feature to provide optimal configurations for these interactions.

https://raw.githubusercontent.com/OpenLiberty/guides-common/prod/gitclone.adoc

Creating a configuration profile for the dev environment

The dev environment is a foundational stage where developers experiment, debug, and refine their code, ensuring an application’s reliability before progressing to subsequent stages.

Navigate to the start directory to begin.

The starting Java project, which you can find in the start directory, is a multi-module Maven project comprised of the system and query microservices. Each microservice is in its own corresponding directory, system and query.

system/pom.xml

link:finish/system/pom.xml[role=include]

query/pom.xml

link:finish/query/pom.xml[role=include]

In software processes, there are different setups for tasks such as development, testing, and going live. Making development the starting setup is a practical approach. It helps smooth out the workflow by ensuring immediate access to development-specific resources without requiring additional setup.

The system microservice contains the three build profiles: dev, test, and prod, in which the dev profile is set as the default.

MicroProfile Config’s configuration profile feature allows for the supply of configurations for different environments while only a single profile is active. The active profile is set using the mp.config.profile property, which acts as a unique identifier for each configuration profile and can be set in any of the configuration sources or during the application startup. When a profile is active, its associated configuration properties are used. For the query service, the mp.config.profile property is set to dev in its Maven pom.xml as the default configuration profile.

When you run Open Liberty in dev mode, the dev mode listens for file changes and automatically recompiles and deploys your updates whenever you save a new change.

Open a command-line session and run the following commands to navigate to the system directory and start the system service in dev environment:

cd system
mvn liberty:dev

Open another command-line session and run the following commands to navigate to the query directory and start the query service in dev environment:

cd query
mvn liberty:dev

After you see the following message, your Liberty instance is ready in dev mode:

**************************************************
*     Liberty is running in dev mode.

Dev mode holds your command-line session to listen for file changes. Open another command-line session to continue, or open the project in your editor.

microprofile-config.properties

link:start/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

In the dev environment, the dev configuration profile in the system/pom.xml file is used for running the system service. The system service runs on http port 9081 and https port 9444 using the context root system/dev. It uses a basic user registry with username alice and password alicepwd for resource authorization. Note that the basicRegistry element is a simple case for learning purposes. For more information on user registries, see the User registries documentation.

Point your browser to the http://localhost:9085/query/systems/localhost URL. The query service returns the message: {"fail":"Failed to reach the client localhost."}. This is because the current query service uses the default properties in the query/src/main/resources/META-INF/microprofile-config.properties file to access the system service.

For proper communication with the development system service, the query service should set up a dev configuration profile.

There are two ways to approach this. The first is at the property level: creating configuration profiles for individual properties is useful when only a few number of settings need to be set differently for each CI/CD stage, such as database connection settings for different environments. Alternatively, it can be particularly useful to create configuration profiles in higher-level configuration sources when you need to manage a large number of configuration properties across different environments, such as varying ports and context roots.

Configuring properties at the property level

This approach involves directly specifying property values within the default microprofile-config.properties configuration file. To provide a configuration property for a particular config profile, use the %<config_profile_id>.<property_name>=<value> syntax, where <config_profile_id> is the unique identifier for the configuration profile and <property_name> is the name of the property you want to set.

Replace the microprofile-config.properties file.
query/src/main/resources/META-INF/microprofile-config.properties

microprofile-config.properties

link:finish/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

system/pom.xml

link:finish/system/pom.xml[role=include]

Configure the %dev.* properties in the microprofile-config.properties file based on the values from the dev profile of the system service.

Because the active profile is set to dev, each %dev.* property will override the value of its original property. For example, the %dev.system.httpsPort property will override the system.httpsPort property and the value will be resolved to 9444 in this case.

Because you are running the query service in dev mode, the changes that you made were automatically picked up. Try out the application at the http://localhost:9085/query/systems/localhost URL. You can see the current OS and Java version in JSON format.

Configuring properties using higher-level configuration sources

Creating configuration profiles in higher-level configuration sources offers a structured way to manage extensive configurations. This can be done by creating a configuration file for each profile with the naming convention microprofile-config-<config_profile_id> in the META-INF folder on the classpath, where <config_profile_id> is the unique identifier for the configuration profile. Once you have created the file, you can add your configuration properties to it with the standard <property_name>=<value> syntax.

Create the microprofile-config-dev.properties file.
query/src/main/resources/META-INF/microprofile-config-dev.properties

microprofile-config-dev.properties

link:finish/query/src/main/resources/META-INF/microprofile-config-dev.properties[role=include]

system/pom.xml

link:finish/system/pom.xml[role=include]

Define the system.* properties in the microprofile-config-dev.properties file based on the values from the dev profile of the system service.

Replace the microprofile-config.properties file.
query/src/main/resources/META-INF/microprofile-config.properties

microprofile-config.properties

link:finish/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

server.xml

link:finish/system/src/main/liberty/config/server.xml[role=include]

Remove the %dev.* properties from the microprofile-config.properties file.

Because the active profile is set to dev, the microprofile-config-dev.properties file is loaded on top of the default microprofile-config.properties file. Any system. properties specified in the microprofile-config-dev.properties take precedence over system. property values in the`microprofile-config.properties`.

Now, point your browser to the http://localhost:9085/query/systems/localhost URL to check out the application again. You see the current OS and Java version in JSON format.

When you are done checking out the application in dev environment, exit dev mode by pressing CTRL+C in the command-line sessions where you ran the system and query services.

Creating a configuration profile for the test environment

In CI/CD, the test environment is where integration tests come to life, checking software readiness. A good testing configuration not only ensures smooth operations but also aligns the environment closely with potential production settings.

Create the microprofile-config-test.properties file.
query/src/main/resources/META-INF/microprofile-config-test.properties

microprofile-config-test.properties

link:finish/query/src/main/resources/META-INF/microprofile-config-test.properties[role=include]

system/pom.xml

link:finish/system/pom.xml[role=include]

Define the system.* properties in the microprofile-config-test.properties file based on the values from the test profile of the system service.

Create the QueryEndpointIT class.
query/src/test/java/it/io/openliberty/guides/query/QueryEndpointIT.java

QueryEndpointIT.java

link:finish/query/src/test/java/it/io/openliberty/guides/query/QueryEndpointIT.java[role=include]

Implement endpoint tests to test the basic functionality of the query microservice. If a test failure occurs, you might have introduced a bug into the code.

See the following descriptions of the test cases:

• testQuerySystem() verifies the /query/systems/{hostname} endpoint.

• testUnknownHost() verifies that an unknown host or a host that does not expose their JVM system properties is correctly handled with a fail message.

Running the tests in the test environment

Now, navigate to the start directory.

scripts/testApp.sh|.bat

link:finish/scripts/testApp.sh[role=include]

microprofile-config-test.properties

link:finish/query/src/main/resources/META-INF/microprofile-config-test.properties[role=include]

Test the application under the test environment by running the following script that contains different Maven goals to build, start, test, and stop the services.

https://raw.githubusercontent.com/OpenLiberty/guides-common/prod/os-tabs.adoc

./scripts/testApp.sh

scripts\testApp.bat

If the tests pass, you see output similar to the following example:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.system.SystemEndpointIT
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.539 s - in it.io.openliberty.guides.system.SystemEndpointIT

Results:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

...

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.query.QueryEndpointIT
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.706 s - in it.io.openliberty.guides.query.QueryEndpointIT

Results:

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0

Next steps

Deploying the application to a Kubernetes environment using the Open Liberty Operator is an optional step in this guide.

To further explore deploying microservices using Kubernetes and the Open Liberty Operator, you can read the following guides:

• Deploying a microservice to Kubernetes using Open Liberty Operator

• Deploying a microservice to OpenShift 4 using Open Liberty Operator

A secure production environment is essential to application security. In the previous sections, you learned how to externalize credentials and other properties for accessing the system service by using MicroProfile Config. This strategy makes the application more adaptable to different environments without changing code or configuration files.

In the this section, you’ll learn how to use Kubernetes secrets to provide the credentials and how to pass them to the query service by using MicroProfile Config.

Deploying the application in the prod environment with Kubernetes

deploy.yaml

link:finish/deploy.yaml[role=include]

system/Dockerfile

link:finish/system/Dockerfile[role=include]

query/Dockerfile

link:finish/query/Dockerfile[role=include]

query/pom.xml

link:finish/query/pom.xml[role=include]

microprofile-config.properties

link:finish/query/src/main/resources/META-INF/microprofile-config.properties[role=include]

Before deploying, create the Dockerfile files for both system and query microservices. Then, build their .war files and Docker images in the start directory.

mvn -P prod clean package
docker build -t system:1.0-SNAPSHOT system/.
docker build -t query:1.0-SNAPSHOT query/.

The Maven clean and package goals can clean the target directories and build the .war application files from scratch. The microprofile-config-dev.properties and microprofile-config-test.properties of the query microservice are excluded from the prod build. The default microprofile-config.properties file is automatically applied.

The Docker build commands package the .war files of the system and query microservices with their default configuration into your Docker images.

After building the images, you can create a Kubernetes secret for storing sensitive data such as credentials.

kubectl create secret generic sys-app-credentials \
        --from-literal username=[username] \
        --from-literal password=[password]

For more information about managing secrets, see the Managing Secrets using kubectl documentation.

Finally, write up the deploy.yaml deployment file to configure the deployment of the system and query microservices by using the Open Liberty Operator. The sys-app-credentials Kubernetes secrets set the environment variables DEFAULT_USERNAME and DEFAULT_PASSWORD for the system microservice, and SYSTEM_USER and SYSTEM_PASSWORD for the query microservice.

deploy.yaml

link:finish/deploy.yaml[role=include]

If you want to override another property, you can specify it in the env sections of the deploy.yaml file. For example, set the CONTEXT_ROOT environment variable in the system deployment and the SYSTEM_CONTEXTROOT environment variable in the query deployment.

Once the images and the secret are ready, you can deploy the microservices to your production environment with Kubernetes.

kubectl apply -f deploy.yaml

Great work! You’re done!

You just learned how to use the MicroProfile Config’s configuration profile feature to configure your application for multiple CI/CD environments.

Feel free to try one of the related guides. They demonstrate new technologies that you can learn to expand on what you built in this guide.

https://raw.githubusercontent.com/OpenLiberty/guides-common/prod/attribution.adoc