-
Notifications
You must be signed in to change notification settings - Fork 176
Mavenizing an Ivy Project
If you are new to Maven, there is an 85min presentation on Box called Maven for Pentaho. It's a couple years old, and some of the stated naming conventions and parent pom structures have changed, but it still has instructional value in understanding Maven.
The end-goal is for a complete ivy-to-maven conversion is to be able to run
$ mvn clean install
and produce ALL SNAPSHOT artifacts required that go into our releases.
Please refer to our standards on maven folder structure when migrating a project's files. Maven works best when you follow its well defined conventions rather than trying to configure it to your desires.
Always create a separate branch based on upstream/master for your mavenization work.
Don't forget to add a README.md file at the root level of your project if one doesn't already exist. Make sure you add content that gives a community member or new developer enough information to be able to build the project.
Start with a really basic pom.xml like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.pentaho</groupId>
<artifactId>acme-project</artifactId>
<version>7.1-SNAPSHOT</version>
<packaging>jar</packaging>
<scm>
<developerConnection>scm:git:git@github.com:pentaho/acme-project.git</developerConnection>
<tag>HEAD</tag>
</scm>
</project>
The GAV values ${groupId}, ${artifactId}, and ${version} should come from what has been specified in the build.properties or assembly.properties of your project. For example, if you have
project.revision=7.x-SNAPSHOT
ivy.artifact.group=org.pentaho
ivy.artifact.id=pentaho-database-model
Then your pom.xml would contain
<groupId>org.pentaho</groupId>
<artifactId>pentaho-database-model</artifactId>
<version>7.x-SNAPSHOT</version>
The element is a specific requirement of the maven-release-plugin. While we will most likely NOT be using this plugin for the releasing of most artifacts, it IS extremely helpful for documentation purposes to know where code came from.
The packaging is usually either going to be "jar" or "pom", but "bundle" and "kar" get usage as well. If the primary artifact that will be generated is a jar then
<packaging>jar</packaging>
Assemblies usually have a packaging type of pom
<packaging>pom</packaging>
Please note, declaring a package type of "bundle" should only be considered in the CE case. The complexities of obfuscation in EE projects cause the default phase bindings of "bundle" to do more harm than good. The explicit bindings in the pentaho-ee-bundle-parent-pom work best when you simply declare a package type of "jar".
Dependency version properties can be referenced here: https://github.com/pentaho/build-resources/blob/7.1-qat/version.properties
Dependencies in Maven are declared in a element and take the form:
<dependency>
<groupId></groupId>
<artifactId></artifactId>
<version></version>
<classifier></classifier>
<type></type>
<scope></scope>
</dependency>
The GAV should be pretty straight-forward. However, due to the requirement of the Buildteam to set versions dynamically, we have established the following requirements regarding versions.
For more detailed information on dependency management, refer to the page on Maven POM Dependency Management.
NOTE: Buildteam will be modifying the specific versions of Pentaho projects and dependencies at release build time. No 3rd party version will be set at build release time.
Make sure you familiarize yourself with http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
| Scope | Description |
|---|---|
| compile | the dependency classes are directly imported into the source code and/or source code generation utilities |
| runtime | the dependency classes are required at runtime but are not directly referenced within the source code |
| provided | the dependency is of compile or runtime scope, but is provided by the environment in which the artifact will run; you may also consider using this to ensure that a declared dependency will not be resolved transitively by any other artifacts dependent upon this one |
| test | the dependency classes are only required for unit or integration testing |
| system | WARNING!!! please consult buildteam@pentaho.com if you ever feel compelled to use this; there are probably better ways to achieve your goal |
It is expected that dependencies will be kept grouped together by scope (compile, runtime, provided, and test) and then ordered alphabetically by groupId and artifactId.
<dependencies>
<!-- ######################### COMPILE ######################### -->
<!-- compile dependencies go here -->
<!-- ######################### RUNTIME ######################### -->
<!-- runtime dependencies go here -->
<!-- ######################### PROVIDED #########################-->
<!-- provided dependencies go here -->
<!-- ######################### TEST ######################### -->
<!-- test dependencies go here -->
</dependencies>
All Ivy configurations will need to be mapped to one of the (4) Maven scopes: compile, runtime, provided, and test. Pay close attention to dependencies that have declared transitive="false". These dependencies will need to make use of wildcard exclusions in Maven. Here is an example:
Ivy Dependency
<dependency org="commons-logging" name="commons-logging" rev="1.1.1" transitive="false">
Maven Dependency
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
As an initial pass at creating the dependencies in a pom file, its advisable to start with the same classpath(s) as Ivy. Maven tools from within an IDE (Eclipse, IntelliJ, Netbeans, etc) can be used to compare the Ivy resolved artifacts ("ant resolve") with the Maven resolved artifacts.
** IMG **
Some IDE(s) will show you the entire Maven dependency tree visually. However, you can also see it from the command line:
** IMG **
Ivy now has the ability to generate a pom.xml file that is sensitive to transitive resolution directives. Its a great way to generate an initial dependencies section in your pom that ought to perform similarly to the Ivy resolve. After doing so, you are highly encouraged to read Maven POM Dependency Management and try to clean up unused dependencies as much as possible.
To generate a pom.xml from an Ivy project configured with subfloor:
$ ant create-pom
Create your assemblies respecting the project structure guidelines and artifact conventions and always make sure that the new generated artifacts provide the same contents than the old ones. Any changes should be intentional and documented within the Jira case for future reference.
Sometimes there is a need to temporarily move/create files during the assembly or testing phases. These files should be located beneath the target directory, ${project.build.directory}. This will allow these files to be cleaned up during the build and avoid leaving uneccessary artifacts on the build file system
Report generation typically happens on a per module basis. In a multi-module build, the parent pom declaring the child modules gets build first, including any reporting bound to it. Reporting plugins have to be specifically crafted to understand and process an aggregation of child reports back up into the parent after the child modules have been executed. This is basically post processing after submodule execution. In situations where you would like an aggregate view, it is unwise to invoke the publishing of reporting results to Sonar in-band. Sonar has limited capabilities for handling and understanding aggregate publishing. Invoking Sonar as part of the regular build will cause Sonar to attempt a publish during every module execution. The end result, if you've configured Sonar at the parent level to publish to a particular name and branch, will be that Sonar ends up being continually overridden by submodules until the last child module is built. In short, your Sonar results will only reflect that last module executed.
This is why we invoke Sonar out-of-band, and after the reporting modules have generated aggregate views as part of the "site" phase.
Publishing to Sonar is done by activating the sonar profile, invoking the sonar goal, and referencing the execution ID in the Pentaho parent pom that is configured for it. (e.g. mvn -Dsonar sonar:sonar@sonar_publish )
You must specifically add report aggregation to your top level project parent pom where you want the aggregation to happen. The following directives are known to work:
<profile>
<id>aggregate-reporting</id>
<activation>
<property>
<name>!skipTests</name>
</property>
</activation>
<reporting>
<plugins>
<plugin>
<artifactId>maven-jxr-plugin</artifactId>
<reportSets>
<reportSet>
<id>aggregate-jxr</id>
<reports>
<report>aggregate</report>
</reports>
<inherited>false</inherited>
</reportSet>
</reportSets>
<configuration>
<linkJavadoc>true</linkJavadoc>
</configuration>
</plugin>
<plugin>
<artifactId>maven-javadoc-plugin</artifactId>
<reportSets>
<reportSet>
<id>aggregate-javadoc</id>
<reports>
<report>aggregate</report>
</reports>
<inherited>false</inherited>
</reportSet>
</reportSets>
<configuration>
<failOnError>false</failOnError>
</configuration>
</plugin>
<plugin>
<artifactId>maven-checkstyle-plugin</artifactId>
<reportSets>
<reportSet>
<id>aggregate-checkstyle</id>
<reports>
<report>checkstyle-aggregate</report>
</reports>
<inherited>false</inherited>
</reportSet>
</reportSets>
<configuration>
<configLocation>${checkstyle-config-url}</configLocation>
<propertiesLocation>${checkstyle-properties-url}</propertiesLocation>
<linkXRef>true</linkXRef>
<cacheFile />
</configuration>
</plugin>
</plugins>
</reporting>
</profile>
Some projects, but not all, generate an WADL xml file that is included in the jar (META-INF/wadl folder). One example of this is data-access. This can't be triggered in a common fashion due to maven limitations on how profiles are triggered. So, if you need to generate a wadl and place it in the resulting jar use this template profile (tweak as required) to accomplish it...
<profile>
<id>wadl-generation</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.10.4</version>
<executions>
<execution>
<goals>
<goal>javadoc</goal>
</goals>
<phase>compile</phase>
</execution>
</executions>
<configuration>
<encoding>UTF-8</encoding>
<verbose>false</verbose>
<show>public</show>
<subpackages>${wadl.package}</subpackages>
<doclet>org.pentaho.wadl.PentahoResourceDoclet</doclet>
<docletPath>${path.separator}${project.build.outputDirectory}</docletPath>
<docletArtifacts>
<docletArtifact>
<groupId>pentaho</groupId>
<artifactId>pentaho-platform-build-utils</artifactId>
<version>${project.version}</version>
</docletArtifact>
<docletArtifact>
<groupId>com.sun.jersey.contribs</groupId>
<artifactId>wadl-resourcedoc-doclet</artifactId>
<version>${jersey.version}</version>
</docletArtifact>
<docletArtifact>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<version>${jersey.version}</version>
</docletArtifact>
<docletArtifact>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
<version>${xercesImpl.version}</version>
</docletArtifact>
</docletArtifacts>
<!-- the following option is required as a work around for
version 2.5 of the javadoc plugin which will be used
by a maven version > 2.0.9 -->
<useStandardDocletOptions>false</useStandardDocletOptions>
<additionalparam>-output ${wadl.path}</additionalparam>
</configuration>
</plugin>
<!--copy the generated wadl to the META-INF/wadl folder for the jar-->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>2.5</version>
<executions>
<execution>
<id>copy-resources</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${wadl.outupt.directory}</outputDirectory>
<resources>
<resource>
<directory>${project.build.directory}</directory>
<filtering>false</filtering>
<includes>
<include>${wadl.file.name}</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
Examples of the properties that this uses:
<properties>
<wadl.file.name>wadlExtension.xml</wadl.file.name>
<wadl.path>${project.build.directory}/${wadl.file.name}</wadl.path>
<wadl.package>org.pentaho.platform.dataaccess</wadl.package>
<wadl.outupt.directory>${project.build.outputDirectory}/META-INF/wadl</wadl.outupt.directory>
<xercesImpl.version>2.9.1</xercesImpl.version>
<jersey.version>1.19.1</jersey.version>
</properties>
Dependency order considerations require Pentaho pillar projects: Reporting, Kettle, Mondrian, and of course the Pentaho platform code must be able to split out sub-modules, or a tree of sub-modules to build independently at various points in a suite build.
Any future mavenization of such projects should only be undertaken with the advise and consent of build team members.
Parent Pom Plugin Operation may provide some assistance in understanding the default ordering of phase bound plugin goals in your project. If you are having build problems, you should pay close attention to your output and ensure you understand what is happening at each phase of execution. With so many profiles and inherited parent poms, it can be difficult to predict the exact execution order of goals in a single phase. Having an understanding of each step will aide significantly in your ability to comprehend and fix execution ordering issues. mvn help:active-profiles and mvn help:effective-pom are also quite helpful.