Drop P2 ant tasks #383

laeubi · 2023-11-08T09:42:23Z

laeubi
Nov 8, 2023
Maintainer

I just noticed that p2 build and ship some ant tasks in director.app for example (just search for jars.extra.classpath in p2).

I wonder if any of these are still useful today or if we can drop them to reduce the dependency / technology stack of p2?

org.eclipse.equinox.p2.director.app.ant.DirectorTask
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntBasedProcessorExecutor
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntSignCommand
org.eclipse.equinox.internal.p2.jarprocessor.ant.JarProcessorTask
org.eclipse.equinox.internal.p2.publisher.ant.FeaturesAndBundlesPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.ProductPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.PublisherTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CompositeRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CreateCompositeArtifactRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorArtifactsTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorMetadataTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ProcessRepoTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.RemoveIUTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.Repo2RunnableTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ValidateTask

Should we drop p2 ant support

Yes

33%

No

50%

I don't care

16%

6 votes

akurtakov · 2023-11-08T09:49:11Z

akurtakov
Nov 8, 2023
Collaborator

While I don't use any of the p2 tasks nor see usage of them it would be nice to ask people to vote here at cross-project-issue-dev so possible missing functionality in Tycho stack could be identified. If one votes for keeping it and is not actively helping in maintenance he/she should provide a viable reason to do so.

0 replies

merks · 2023-11-08T10:18:36Z

merks
Nov 8, 2023
Collaborator

Is that what's being used here?

0 replies

cdietrich · 2023-11-08T10:19:27Z

cdietrich
Nov 8, 2023

how can one find out if they are affected?

11 replies

ewillink Nov 8, 2023

I don't think that works. Your more direct use of p2.composite.repository seems the same as my less direct use via /shared/modeling/tools/promotion/manage-composite.xml. But this is not sensibly done in Tycho since the Tycho/Maven build doesn't run on projects-storage.eclipse.org. Now that SSH access has gone I have to use a separate promoter job that does the equivalent of

ssh genie.ocl@projects-storage.eclipse.org cd /home/data/httpd/download.eclipse.org/modeling/mdt/ocl/updates/nightly ; export JAVA_HOME=/shared/common/jdk1.8.0_x64-latest ; java -version ; /shared/common/apache-ant-latest/bin/ant -f /shared/modeling/tools/promotion/manage-composite.xml remove -Dchild.repository=6.12.0

A new P2 tool, especially one with a web-interface accessed on the standard '404' EF browser page could be wonderful.

LorenzoBettini Nov 8, 2023

We use the Ant tasks (https://github.com/eclipse/xtext/blob/main/org.eclipse.xtext.p2repository/packaging-p2composite.ant) to build the composite update site; we run those tasks through the tycho-eclipserun-plugin (https://github.com/eclipse/xtext/blob/main/org.eclipse.xtext.p2repository/pom.xml#L210). To the best of my knowledge, those p2 ant tasks are the only way to manage composite update sites (besides manually editing XML files, of course). I don't think there's anything in Tycho with that respect.

A few months ago, I started to work on a Maven plugin for managing p2 composite sites: https://github.com/LorenzoBettini/p2composite-maven-plugin.

It's functional and tested, but I still need to release it. I was also planning to propose it to be part of Tycho.

laeubi Nov 8, 2023
Maintainer Author

A few months ago, I started to work on a Maven plugin for managing p2 composite sites: https://github.com/LorenzoBettini/p2composite-maven-plugin.
It's functional and tested, but I still need to release it. I was also planning to propose it to be part of Tycho.

That would be great, such tasks should not require to use antrun or similar...

mickaelistria Nov 8, 2023
Collaborator

Actually, such "infra" work that is not really artifact-first should even not require Tycho. IMO those Ant task would fit best as Eclipse applications so they could be automated in bash; and Tycho or other could relatively easily consume them.

HannesWell Nov 8, 2023
Collaborator

To the best of my knowledge, those p2 ant tasks are the only way to manage composite update sites (besides manually editing XML files, of course). I don't think there's anything in Tycho with that respect.

I wanted to mention the same. IIRC RemoveIUTask is also used somewhere.
As long as there is no Maven/Tycho or application way to manipulate Composite-Repos or in genera more advanced ways to e.g. filter repo content like it would e.g. have been required in eclipse-m2e/m2e-core#1592 I think the ANT-tasks have to stay.
Nevertheless I would appreciate if there were the mentioned alternatives, that are also used in the Eclipse Releng work.

iloveeclipse · 2023-11-08T10:39:16Z

iloveeclipse
Nov 8, 2023
Collaborator

p2 build and ship some ant tasks

Please provide a summary of tasks you are talking about. It is really the minimal requirement before creating a request to remove them.

1 reply

laeubi Nov 8, 2023
Maintainer Author

As mentioned any task with p2 in its name...

This is the list I can find by looking at p2 projects

org.eclipse.equinox.p2.director.app.ant.DirectorTask
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntBasedProcessorExecutor
org.eclipse.equinox.internal.p2.jarprocessor.ant.AntSignCommand
org.eclipse.equinox.internal.p2.jarprocessor.ant.JarProcessorTask
org.eclipse.equinox.internal.p2.publisher.ant.FeaturesAndBundlesPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.ProductPublisherTask
org.eclipse.equinox.internal.p2.publisher.ant.PublisherTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CompositeRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.CreateCompositeArtifactRepositoryTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorArtifactsTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorMetadataTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.MirrorTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ProcessRepoTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.RemoveIUTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.Repo2RunnableTask
org.eclipse.equinox.p2.internal.repository.tools.tasks.ValidateTask

ewillink · 2023-11-08T10:42:12Z

ewillink
Nov 8, 2023

In principle I agree. EMF, OCL and QVTo continue to provide some Ant tasks and it often seems that it takes more developer time to maintain them than it saves in misguided user time. If platform deprecates Ant then we can simplify.

But you are wrong to write that Tycho replaces Ant. You ignore the 'External Tools Configuration' run that has many legacy scripts. However IMHO these are much better once replaced by MWE2, so just revise your comment to write that Ant is replaced by Tycho or MWE2.

BUT in 10 years time is it more likely that Ant or Tycho will still be working? Buckminster didn't last long; I suspect Tycho will fade way quite soon too. An Eclipse without Ant seems misguided; of course work on integrating the Ant taskpath / debugging is long overdue.

0 replies

mickaelistria · 2023-11-08T11:00:09Z

mickaelistria
Nov 8, 2023
Collaborator

If you search <p2. in all *.xml files; you'll see a lot of usage of those tasks in various scripts used by Platform. So as long as there are users of those tasks, particularly in Platform, it's too early to consider removing them.

4 replies

laeubi Nov 8, 2023
Maintainer Author

As said any legacy stuff can simply use the ones as deployed with the current and previous or any ancient releases.... some of these scripts even look outdated as the mention pack and md5 what is already no longer supported by p2 anyways

mickaelistria Nov 8, 2023
Collaborator

As said any legacy stuff can simply use the ones as deployed with the current and previous or any ancient releases....

That's too dangerous to use unmaintained bits in the release engineering process. What if one of those tasks is discovered to have a major CVE and we've dropped code? Then we can't just fix it and getting rid of the CVE would require to rework the whole existing scripts to something more modern; which would then become a lot of trouble.
As long as those are used, we must provide an easy maintenance path. Keeping the code in current master does that.

laeubi Nov 8, 2023
Maintainer Author

As long as those are used, we must provide an easy maintenance path. Keeping the code in current master does that.

It does not, as seen with recent PDE changes, where no one really feels comfortable to maintain/fix this stuff, so without anyone being dedicated to actually maintain that code it is effectively dead and risk of preventing further improvements and maintenance...

akurtakov Nov 8, 2023
Collaborator

For reference, usages I've found are:
cje-production/cleaners/cleanupNightlyRepo.sh: echo -e " <p2.composite.repository>" >>$antBuildFile
cje-production/scripts/addToComposite.xml: <p2.composite.repository>
cje-production/scripts/removeFromComposite.xml: <p2.composite.repository>
cje-production/scripts/updateGenericComposites.xml: <p2.composite.repository>
eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/process-artifacts.xml: <p2.process.artifacts
eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/process-artifacts.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/eclipse/buildScripts/process-artifacts.xml: <p2.mirror
eclipse.platform.releng.tychoeclipsebuilder/equinox/buildConfigs/equinox-launchers/build.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/equinox/buildConfigs/equinox-launchers/build.xml: <p2.repo2runnable destination="${featureTemp2}/eclipse">
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/build.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.remove.iu>
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.mirror source="file://${buildRepo}">
eclipse.platform.releng.tychoeclipsebuilder/repos/platformrepo.xml: <p2.remove.iu>
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.mirror
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.repo2runnable destination="${locationcompare}">
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.mirror
eclipse.platform.releng/bundles/org.eclipse.releng.tests/test.xml: <p2.repo2runnable destination="${locationcomparecurrent}">

ewillink · 2023-11-08T11:13:46Z

ewillink
Nov 8, 2023

/shared/modeling/tools/promotion/manage-composite.xml is the technology I continue to use via a 'shell' job to maintain composite release repos. Jenkins 'promoter' jobs also ujse it. It uses

<target name="-add" depends="-add.named" unless="composite.name"> <p2.composite.repository> <repository location="${user.dir}" /> <add> <repository location="${child.repository}" /> </add> </p2.composite.repository> </target>

so until the EF provides a web-access P2 maintainance tool or relents on the removal of ssh access, I suspect that p2.composite.repository is required.

0 replies

jonahgraham · 2023-11-08T18:19:05Z

jonahgraham
Nov 8, 2023

I don't have strong opinion on the removal (although the general sense reading these comments is that there are still many uses of it).

However I assume that removing this would follow the normal removal guidelines, i.e. deprecated & announced and wait two years before actual removal?

1 reply

laeubi Nov 9, 2023
Maintainer Author

However I assume that removing this would follow the normal removal guidelines, i.e. deprecated & announced and wait two years before actual removal?

I think so yes, even if it is not really an "API", one could consider this as user visible interface also the only reference I found is here: https://wiki.eclipse.org/Equinox/p2/Ant_Tasks what claims to be "maybe outdated" (as of 2009 (!) )....

cdietrich · 2023-11-09T05:53:01Z

cdietrich
Nov 9, 2023

how do other project manage their composite update sites?!?

3 replies

merks Nov 9, 2023
Collaborator

Like Xpect:

https://github.com/eclipse/Xpect/blob/0343856120f1cff90b55ede16d2c0f0debb2676d/org.eclipse.xpect.releng/promotion/pom.xml#L27-L111

And DataTools:

https://github.com/eclipse-datatools/datatools/blob/22b1b935815888c5092aefc202cea28bfb18ac8f/promotion/pom.xml#L20-L108

And EMF:

https://github.com/eclipse-emf/org.eclipse.emf/blob/master/releng/org.eclipse.emf.parent/promotion/pom.xml

And so on...

cdietrich Nov 9, 2023

thx have doubts i can work on this any time soon. maybe @LorenzoBettini can

LorenzoBettini Nov 9, 2023

@cdietrich I think we can stay with the current solution since those Ant tasks will not be removed soon (that is what I understand from other comments). Moreover, for those specific tasks, one could still use the versions from older p2 repositories (we just need to point to an old Eclipse repository in the POM). Thus, we're definitely safe ;)

However, I'll keep the above links as references for the future.

But, if we had to go with an alternative, I'd use my Maven plugin (not yet published), which might be easier to use.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop P2 ant tasks #383

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 9 comments 20 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Drop P2 ant tasks #383

laeubi Nov 8, 2023 Maintainer

Replies: 9 comments · 20 replies

akurtakov Nov 8, 2023 Collaborator

merks Nov 8, 2023 Collaborator

laeubi Nov 8, 2023 Maintainer Author

mickaelistria Nov 8, 2023 Collaborator

HannesWell Nov 8, 2023 Collaborator

iloveeclipse Nov 8, 2023 Collaborator

laeubi Nov 8, 2023 Maintainer Author

mickaelistria Nov 8, 2023 Collaborator

laeubi Nov 8, 2023 Maintainer Author

mickaelistria Nov 8, 2023 Collaborator

laeubi Nov 8, 2023 Maintainer Author

akurtakov Nov 8, 2023 Collaborator

laeubi Nov 9, 2023 Maintainer Author

merks Nov 9, 2023 Collaborator

laeubi
Nov 8, 2023
Maintainer

Replies: 9 comments 20 replies

akurtakov
Nov 8, 2023
Collaborator

merks
Nov 8, 2023
Collaborator

laeubi Nov 8, 2023
Maintainer Author

mickaelistria Nov 8, 2023
Collaborator

HannesWell Nov 8, 2023
Collaborator

iloveeclipse
Nov 8, 2023
Collaborator

laeubi Nov 8, 2023
Maintainer Author

mickaelistria
Nov 8, 2023
Collaborator

laeubi Nov 8, 2023
Maintainer Author

mickaelistria Nov 8, 2023
Collaborator

laeubi Nov 8, 2023
Maintainer Author

akurtakov Nov 8, 2023
Collaborator

laeubi Nov 9, 2023
Maintainer Author

merks Nov 9, 2023
Collaborator