[HUDI-7596] Enable Jacoco code coverage report across multiple modules

[danny0405]

Change Logs

Enable the Jacoco code coverage for Azure CI.

Impact

none

Risk level (write none, low medium or high below)

none

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[danny0405]

@hudi-bot run azure

[yihua]

Can we try adding report-aggregate goal somewhere to report the coverage across modules? Right now it does not seem like the report is aggregated across modules.

                        <execution>
                            <id>report</id>
                            <phase>prepare-package</phase>
                            <goals>
                                <goal>report-aggregate</goal>
                            </goals>
                        </execution>

[yihua]

This is the example: https://github.com/jacoco/jacoco/blob/master/jacoco-maven-plugin.test/it/it-report-aggregate/report/pom.xml

[danny0405]

I tried that and it reports nothing, probably because we use the v2 reportage plugin which is recommended, let me tweak and have a try again.

[danny0405]

It looks like it is very tricky to make the report-aggregate work, based on these links:

• https://groups.google.com/g/jacoco/c/FpdLbxsXSTY
• https://github.com/jacoco/jacoco/blob/master/jacoco-maven-plugin.test/it/it-report-aggregate/report/pom.xml

Basically if we wanna to make the aggrerator work, we should have a separate module named hudi-reporter that adds the correspendent denpendencies for the multiple moldules that we want to report, otherwise, we always got empty report files.

But some modules are actually dinamically loaded based on the active maven compile profile, which makes the issue even more complicated.

[danny0405]

I took several attempts like let each sub-module have a report-aggregate gole, but that still reports 0 coverage. So revert the code back to the non-aggregate version.

Based on this doc https://github.com/jacoco/jacoco/wiki/MavenMultiModule, we should choose the third strategy that create a delegate module with all the dependencies, and here is a demo project: https://github.com/jacoco/jacoco/tree/master/jacoco-maven-plugin.test/it/it-report-aggregate, I followed the instructions step by step but still got nothing.

[yihua]

I'm wondering if the goal should be verify, i.e., mvn clean verify for report-aggregate to work. Have you tried the exact repo from https://github.com/jacoco/jacoco/blob/master/jacoco-maven-plugin.test/it/it-report-aggregate and see if it works locally with maven?

For the issue of dynamic loaded modules, maybe we can first try excluding such modules like hudi-spark and run for hudi-spark-client for example.

[danny0405]

It turns out to be that we must run the mvn test in the root dir of the project and we can not specify specific modules with -pl option. This is a limination for our Azure CI becase we split the modules into 6 separate job.

[yihua]

One idea is that we can aggregate the report for a particular module and run the mvn test from that module directory. If that works, we can run tests in different modules in a different way without -pl as a compromise.

[danny0405]

Like I said, execute mvn test under module dir does not work.

[hudi-bot]

CI report:

• 39c44a3 UNKNOWN
• d329d24 UNKNOWN
• 7d82c28 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[vinothchandar]

I notice that the azure CI results are now grouped by folders? and could we show an example to illustrate that cross module FT coverage is now factored in?

Could we fix these before landing?

[danny0405]

I notice that the azure CI results are now grouped by folders? and could we show an example to illustrate that cross module FT coverage is now factored in?

Could we fix these before landing?

In order to make the cross module FT coverage work, we need to:

1. add a new separate module that including dependencies of all the modules that we want to report;
2. run all the module tests in one shot and there is no way to run specific module with -pl option;

Our CI tests are split into 6 jobs with each covering different modules, which is not suitable for the jacoco report-aggregate goal.