Download Gradle Plugins from Nexus

[renedupont]

EDIT: This issue was named "Can't locally pull gradle plugins from inside corporate network" beforehand but the plugin URL seems to be whitelisted by now and hence it is not a blocking bug anymore but rather a improvement request.

Describe the bug
It is not possible anymore for us to build locally via gradle because the gradle repository is being blocked by our corporate network (or a service that is being used for security on top). A service application was recently installed on our laptops that adds further security on top of the VPN connection and it might be happening since then, but that is just a guess.

To be more precise, if I use gradle and have a plugin version in my build.gradle that is NOT already locally (cached) on my machine, I get with this part in my build.gradle

plugins {
    id 'org.springframework.boot' version '2.6.6'
    ...
}

the following error:

> Could not resolve all files for configuration ':classpath'.
   > Could not resolve org.springframework.boot:spring-boot-buildpack-platform:2.6.6.
     Required by:
         project : > org.springframework.boot:org.springframework.boot.gradle.plugin:2.6.6 > org.springframework.boot:spring-boot-gradle-plugin:2.6.6
      > Could not resolve org.springframework.boot:spring-boot-buildpack-platform:2.6.6.
         > Could not get resource 'https://plugins.gradle.org/m2/org/springframework/boot/spring-boot-buildpack-platform/2.6.6/spring-boot-buildpack-platform-2.6.6.module'.
            > Could not GET 'https://repo.gradle.org/artifactory/jcenter/org/springframework/boot/spring-boot-buildpack-platform/2.6.6/spring-boot-buildpack-platform-2.6.6.module'. Received status code 403 from server: Forbidden
   > Could not resolve org.springframework.boot:spring-boot-loader-tools:2.6.6.
     Required by:
         project : > org.springframework.boot:org.springframework.boot.gradle.plugin:2.6.6 > org.springframework.boot:spring-boot-gradle-plugin:2.6.6
      > Could not resolve org.springframework.boot:spring-boot-loader-tools:2.6.6.
         > Could not get resource 'https://plugins.gradle.org/m2/org/springframework/boot/spring-boot-loader-tools/2.6.6/spring-boot-loader-tools-2.6.6.module'.
            > Could not GET 'https://repo.gradle.org/artifactory/jcenter/org/springframework/boot/spring-boot-loader-tools/2.6.6/spring-boot-loader-tools-2.6.6.module'. Received status code 403 from server: Forbidden
   > Could not resolve org.springframework:spring-core:5.3.18.
     Required by:
         project : > org.springframework.boot:org.springframework.boot.gradle.plugin:2.6.6 > org.springframework.boot:spring-boot-gradle-plugin:2.6.6
      > Could not resolve org.springframework:spring-core:5.3.18.
         > Could not get resource 'https://plugins.gradle.org/m2/org/springframework/spring-core/5.3.18/spring-core-5.3.18.module'.
            > Could not GET 'https://repo.gradle.org/artifactory/jcenter/org/springframework/spring-core/5.3.18/spring-core-5.3.18.module'. Received status code 403 from server: Forbidden

This issue was raised by a colleague who also found a way to deal with it by not using the public gradle plugin portal but rather going via the corporates nexus. For this to work the repositories have to be added to settings.gradle as described here:
https://docs.gradle.org/current/userguide/plugins.html#sec:custom_plugin_repositories

For this to understand it is important to know that the gradle plugin repositories are different (they are used just for the plugins) than the repositories that are being used by gradle for pulling the application's dependencies and that we already configure in our quickstarter to use nexus (see https://github.com/opendevstack/ods-quickstarters/blob/master/be-java-springboot/templates/gradle-repositories.template)

A solution (next to asking the IT to unblock the public repo) could be to add the nexus plugin repos to settings.gradle if nexus vars are set (analogous to like we do it here) while provisioning.

To Reproduce
Steps to reproduce the behavior:

1. Be behind a network that blocks the public gradle plugin portal
2. Don't have any plugins cached locally that you are going to need for building the blank spring boot qs
3. Provision the spring boot qs component
4. in the projects root folder type: ./gradlew

[tbugfinder]

If nexus gets provisioned from scratch by the current configuration, initial downloads will also fail.

[renedupont]

I tried a few things today and came up with a possible solution, before creating a draft PR I'd like to have feedback and start discussing possible other options/improvements.

My idea would be to basically move the nexus vars handling that we currently have in build.gradle to settings.gradle as part of the pluginManagement. There, we would also set the repository for gradle plugins (see code below). That way, we can reuse the nexus vars in our build.gradle and get rid of the buildscript { ... } block there. Last, the public gradle-plugins repo has to be added to nexus as a proxy repo otherwise we can't download them for there...

My change would include:

• Add the gradle-plugins repo https://plugins.gradle.org/m2/ as a proxy repo to nexus, like the guy did here (I tried it successfully in our nexus aswell).
• During the provisioning, add as first statement of settings.gradle the following code:

pluginManagement {

    // set nexus vars
    gradle.ext.nexus_url = settings.hasProperty('nexus_url') ? settings.nexus_url : System.getenv('NEXUS_HOST')
    gradle.ext.nexus_user = settings.hasProperty('nexus_user') ? settings.nexus_user : System.getenv('NEXUS_USERNAME')
    gradle.ext.nexus_pw = settings.hasProperty('nexus_pw') ? settings.nexus_pw : System.getenv('NEXUS_PASSWORD')
    gradle.ext.no_nexus = (settings.hasProperty('no_nexus') ? settings.no_nexus : System.getenv('NO_NEXUS')
            ?: false).toBoolean()
    gradle.ext.nexusFolderReleases = settings.hasProperty('nexus_folder_releases') ? settings.nexus_folder_releases
            : System.getenv('NEXUS_FOLDER_RELEASES') ?: "maven-releases"
    gradle.ext.nexusFolderSnapshots = settings.hasProperty('nexus_folder_snapshots') ? settings.nexus_folder_snapshots
            : System.getenv('NEXUS_FOLDER_SNAPSHOTS') ?: "maven-snapshots"
    if (!gradle.no_nexus && (gradle.nexus_url == null || gradle.nexus_user == null || gradle.nexus_pw == null)) {
        throw new GradleException("property 'no_nexus' is set to false, but neither 'nexus_url', 'nexus_user' " +
                "nor 'nexus_pw' is configured. You can do so by e.g. creating a gradle.properties file in your " +
                "'GRADLE_USER_HOME' (by default '~/.gradle') folder and setting the nexus properties there.")
    }

    repositories {
        if (gradle.no_nexus) {
            println("using public repository 'gradlePluginPortal()', because property no_nexus=${gradle.no_nexus}")
            gradlePluginPortal()
        } else {
            println("using gradle plugin repositories proxied via nexus")
            def nexusMaven = { repoUrl ->
                maven {
                    credentials {
                        username = "${gradle.nexus_user}"
                        password = "${gradle.nexus_pw}"
                    }
                    url repoUrl
                }
            }
            nexusMaven("${gradle.nexus_url}/repository/gradle-plugins")
        }
    }
}

• Due to the above, we check and set the nexus vars already in an earlier stage, basically before build.gradle gets executed and thereby we can remove the buildscript part in the build.gradle.
• But since we added the nexus vars in settings.gradle via gradle.ext.xyz, we have to access them in build.gradle now by adding a gradle. in front of them in case they are not resolved as a part of a string (don't ask... gradle...). So basically we have to do this for no_nexus, nexusFolderReleases and nexusFolderSnapshots.

[renedupont]

I assume the gradle plugin portal URL got whitelisted by now in our internal network and hence this issue here might not be a bug per se anymore. But, as described above, if we want to use nexus for pulling gradle plugins as well instead of getting them from the public plugin portal, we still need to consider something like my proposal above.
Also, a solution for also downloading gradle wrapper distributions from nexus and not from public on every pipeline run is still missing.
I'll take the bug label off and rename this ticket to "Download gradle plugins from Nexus".