Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fill variables in redisson.yml with Spring Boot values. #3206

Open
broodjetom opened this issue Nov 11, 2020 · 10 comments · May be fixed by #4640
Open

Fill variables in redisson.yml with Spring Boot values. #3206

broodjetom opened this issue Nov 11, 2020 · 10 comments · May be fixed by #4640
Labels

Comments

@broodjetom
Copy link
Contributor

broodjetom commented Nov 11, 2020

Is your feature request related to a problem? Please describe.

We are using redisson-spring-boot-starter. We were hoping that we could configure Redisson with a redisson.yml, but also configure more dynamic properties from our application.yml.

# application.yml
spring:
  redis:
    host: ${REDIS_HOSTNAME}
    redisson: 
      file: classpath:redisson.yaml

When we run in a different environment we can overwrite the Redis hostname by setting an environment variable. With Spring Boot profiles (for example "local"), we could have an application-local.yml that contains the following:

# application-local.yml
spring:
  redis:
    host: localhost

Or set the placeholder itself:

# application-local.yml
REDIS_HOSTNAME: localhost

That way we would be able to have different configurations for our local run configuration compared to what we use in our production systems. Detailed properties for Redisson (described in redisson.yml) are the same for all environment, while things like the host, port or ssl properties can change depending on the environment where the application is running.

Describe the solution you'd like

If the merging of application.yml(or application-local.yml) with redisson.yml is not possible, can we then maybe have some feature where we fill variables with Spring Boot values. That way the redisson.yml would be able to look like:

# redisson.yml
singleServerConfig:
  idleConnectionTimeout: 10000
  connectTimeout: 10000
  timeout: 3000
  retryAttempts: 3
  retryInterval: 1500
  subscriptionsPerConnection: 5
  address: "redis://${REDIS_HOSTNAME}:6379"
  subscriptionConnectionMinimumIdleSize: 1
  subscriptionConnectionPoolSize: 50
  connectionMinimumIdleSize: 10
  connectionPoolSize: 64
  dnsMonitoringInterval: 5000

In such a way, we would be able to create an application-local.yml that looks like this:

# application-local.yml
REDIS_HOSTNAME: localhost

With Spring Boot values I mean the values described by externalized configuration here. That way there are many ways to inject variables into the configuration of Redisson.

Describe alternatives you've considered

We currently are using the following as an application.yml:

# application.yml
spring:
  redis:
    redisson:
      config: |
        singleServerConfig:
          idleConnectionTimeout: 10000
          connectTimeout: 10000
          timeout: 3000
          retryAttempts: 3
          retryInterval: 1500
          subscriptionsPerConnection: 5
          address: "redis://${ELASTICACHE_URL}:6379"
          subscriptionConnectionMinimumIdleSize: 1
          subscriptionConnectionPoolSize: 50
          connectionMinimumIdleSize: 10
          connectionPoolSize: 64
          dnsMonitoringInterval: 5000

This works for us but puts a lot of detail for a specific dependency in the application.yml. These fields are never changed. The values are prone to mistakes.

@mrniko mrniko added this to the 3.13.7 milestone Nov 11, 2020
@mrniko mrniko added the feature label Nov 11, 2020
@broodjetom
Copy link
Contributor Author

I would like to implement this, but I need some more guidance from the maintainers. In the documentation of redisson-spring-boot-starter it states that you can use "Common Spring Boot settings" or "Redisson settings". It would be nice if we can use both and merge them in some order. Or we can add some additional documentation about how the settings are prioritized (Redisson settings at the moment).

We can use common Spring Boot settings to setup Redisson:

spring:
  redis:
    secure: false
    host: localhost
    port: 6379
---
spring:
  redis:
    url: redis://localhost:6379

We can achieve the same with Redisson settings:

spring:
  redis:
    redisson: 
      config: |
        singleServerConfig:
          address: redis://localhost:6379
---
spring:
  redis:
    redisson: 
      # Put the configuration into seperate file:
      file: classpath:redisson.yaml

The common Spring Boot settings allow us to dynamically (via externalized configuration) set the properties for Redisson. But there are limitations: there is only a small set of properties we can set with the common Spring Boot settings, described in org.springframework.boot.autoconfigure.data.redis.RedisProperties.

The Redisson settings do not allow us to dynamically (via externalized configuration) set the properties for Redisson. But it also adds a lot of more possibilities. Just take a look at some of the server configurations (ClusterServersConfig, SingleServerConfig) that are available. These configuration let us configure Redisson very precisely, something that is not possible with the common Spring Boot settings.

Can we combine the best of the two settings options: externalized configuration and precise configuration?

I am thinking of the different possibilities: the seperate file for redisson.yaml is nice, but is there some Spring Boot method to include that file into the application configuration? Could we maybe add an additional field to the RedissonProperties so that we more options than only a block (starting with "|") or a file?

Also what should the priority be in merging: The common Spring Boot settings, the Redisson settings. And in turn of Redisson settings, what should overwrite what? We could have file, config, and maybe an additional field.

@mrniko
Copy link
Member

mrniko commented Nov 13, 2020

"Redisson settings" has priority due to limited set of common Spring Boot settings.

but is there some Spring Boot method to include that file into the application configuration?

I'm afraid no.

Could we maybe add an additional field to the RedissonProperties so that we more options than only a block (starting with "|") or a file?

No. Since we need to add all of configuration fields in this case. Which I would like to avoid.

@mrniko mrniko modified the milestones: 3.14.0, 3.14.1 Nov 21, 2020
@mrniko mrniko modified the milestones: 3.14.1, 3.14.2 Dec 22, 2020
@mrniko mrniko modified the milestones: 3.15.0, 3.15.1 Jan 31, 2021
@mrniko mrniko modified the milestones: 3.15.1, 3.15.2 Mar 5, 2021
@mrniko mrniko modified the milestones: 3.15.2, 3.15.3 Mar 13, 2021
@mrniko mrniko modified the milestones: 3.15.3, 3.15.4 Apr 1, 2021
@mrniko
Copy link
Member

mrniko commented Apr 9, 2021

In this case we need to have a direct reference to PropertyResolver object. This makes spring-core as required dependency. Can you suggest better solution?

@mrniko mrniko removed this from the 3.15.4 milestone Apr 9, 2021
@anessi
Copy link

anessi commented Jun 14, 2021

I have implemented the following workaround for this:

  • A RedissonAutoConfigurationCustomizer that stores the config in a static class -> RedissonConfigAccessor
  • A custom RedissonRegionFactory which reads the config object from the RedissonConfigAccessor

Due to the order in which the classes are called it works just fine. However, I consider this more like a workaround than a proper solution. Maybe there's a better way of making the config available in the RedissonRegionFactory.

  @Bean
  public RedissonAutoConfigurationCustomizer redissonConfigCustomizer() {
    return new RedissonAutoConfigurationCustomizer() {

      private String database;

      @SuppressWarnings("synthetic-access")
      @Override
      public void customize(Config config) {
        log.info("Redisson config: {}", ToStringBuilder.reflectionToString(config, new RecursiveToStringStyle()));
        RedissonConfigAccessor.setConfig(config);
      }

    };
  }
public class RedissonConfigAccessor {

  private static Config config;

  public static void setConfig(Config config) {
    RedissonConfigAccessor.config = config;
  }

  public static Config getConfig() {
    return config;
  }

}
public class ConfigEnabledRedissonRegionFactory extends RedissonRegionFactory {

  @Override
  protected RedissonClient createRedissonClient(Map properties) {
    Config config = RedissonConfigAccessor.getConfig();
    log.info("Using the existing redisson config: {}", ToStringBuilder.reflectionToString(config, new RecursiveToStringStyle()));
    if (config == null) {
      throw new IllegalArgumentException(
          "Redisson configuration is null -> expecting configuration to be set");
    }
    return Redisson.create(config);
  }

}

@mrniko
Copy link
Member

mrniko commented Aug 11, 2021

@anessi

How does your solution solves the problem of merging of two configurations - Spring Redis common config and Redisson's?

@anessi
Copy link

anessi commented Sep 3, 2021

I'm relying on the logic in org.redisson.spring.starter.RedissonAutoConfiguration.redisson() for the initial configuration, which is reading the values from org.springframework.boot.autoconfigure.data.redis.RedisProperties. Then I'm customization that configuration to my need in Java using the RedissonAutoConfigurationCustomizer.

I found it quite hard to understand the logic in org.redisson.spring.starter.RedissonAutoConfiguration.redisson() and what is set from where. Without the source code it would be impossible 😄.

IMO the ideal way would be to have a simple POJO that has all the possible Redisson conf values (similar to org.springframework.boot.autoconfigure.data.redis.RedisProperties). This POJO is populated in a defined way, e.g. reading the values from:

  • spring.redis.redisson.config
  • spring.redis.redisson.file
  • org.springframework.boot.autoconfigure.data.redis.RedisProperties

and merging this into the single Redisson properties POJO. Maybe it would make sense to allow either config or file only. The POJO approach similar to org.springframework.boot.autoconfigure.data.redis.RedisProperties would support setting/overriding any value as defined in chapter Externalized Configuration, e.g. using environment variables. The Redisson properties POJO could also be created/modified in another way if needed as it's a normal Spring bean.

This POJO could then be converted to the Redisson Config object. If needed there is still the possibility to customize the result using the RedissonAutoConfigurationCustomizer approach, which is a good thing IMO.

Probably the approach needs a bit more thinking.

@mrniko
Copy link
Member

mrniko commented Sep 4, 2021

I found it quite hard to understand the logic in org.redisson.spring.starter.RedissonAutoConfiguration.redisson() and what is set from where. Without the source code it would be impossible 😄.

Source code deployed in maven repo.
https://repo1.maven.org/maven2/org/redisson/redisson-spring-boot-starter/3.16.2/redisson-spring-boot-starter-3.16.2-sources.jar

IMO the ideal way would be to have a simple POJO that has all the possible Redisson conf values

I would like to find a way to reuse existing Config classes.

@anessi
Copy link

anessi commented Sep 6, 2021

Source code deployed in maven repo.
https://repo1.maven.org/maven2/org/redisson/redisson-spring-boot-starter/3.16.2/redisson-spring-boot-starter-3.16.2-sources.jar

Thanks. I found that, so I was able to understand what is happening.

I would like to find a way to reuse existing Config classes.

I guess you could use a org.redisson.config.Configas the POJO class and then use

@Bean
@ConfigurationProperties(prefix="spring.redis.redisson.config")
public Config redissonConfig() {
    return new Config();
}

Spring would automatically populate all properties using the rules mentioned in the link above.

After having the initial values populated you could 'merge' it with spring.redis.redisson.file and org.springframework.boot.autoconfigure.data.redis.RedisProperties.

@Blockost
Copy link

Blockost commented May 14, 2022

Any news on this @mrniko ? It's been quite some time now since this feature request has been submitted but it does not seem to be implemented yet right ? I'm very interested in this !

@mrniko
Copy link
Member

mrniko commented May 27, 2022

@Blockost

I'm ready to consider any contribution.

broodjetom added a commit to broodjetom/redisson that referenced this issue Oct 29, 2022
broodjetom added a commit to broodjetom/redisson that referenced this issue Oct 29, 2022
… Boot Configuration YAML

Signed-off-by: Tom Scholten <Tom.Scholten@infosupport.com>
broodjetom added a commit to broodjetom/redisson that referenced this issue Oct 29, 2022
broodjetom added a commit to broodjetom/redisson that referenced this issue Oct 29, 2022
… Boot Configuration YAML

Signed-off-by: Tom Scholten <Tom.Scholten@infosupport.com>
broodjetom added a commit to broodjetom/redisson that referenced this issue Oct 29, 2022
…edisson configuration

Signed-off-by: Tom Scholten <Tom.Scholten@infosupport.com>
broodjetom added a commit to broodjetom/redisson that referenced this issue Oct 29, 2022
Signed-off-by: Tom Scholten <Tom.Scholten@infosupport.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
4 participants