automation: Handle Outdated Jobs

[code-sparsh]

Fixes zaproxy/zaproxy#7799

Changes

• The Automation tab reloads the plan and the GUI if an extension of a job present in the plan is removed at runtime.

[kingthorin]

Could use an advanced for and avoid the intermediate storage

[code-sparsh]

Actually I did try to use the advanced for loop. But it was giving me an exception called ConcurrentModificationException, which was telling me to not change the iterator object (planJob). Although, I wasn't changing it but maybe the function which I was passing that on was changing it. So the only solution I could find on the forums was to just switch to the old for loops.

[kingthorin]

Does an advanced for work with the intermediate store? Ex: leave this line assigning to “jobs” then iterate it instead of “plan.getJobs”.

[code-sparsh]

Does an advanced for work with the intermediate store?

I think, no. It is still giving me that exception on run-time.
Is using the traditional for loop a bad practice though?

[kingthorin]

Nope, that’s fine. Just trying to simplify 😉

[code-sparsh]

Have a look on this please.

I have fixed the cases where it was showing errors. After a job is unregistered, it would be removed from the plan as well as the GUI.

I am using some things like sets, keysets to iterate over the LinkedHashMap. It might not be the most optimized way of doing it maybe, I am not sure. I will try to change it a little if you feel it can be little bit more optimized.

Also, as per the comment by @thc202 zaproxy/zaproxy#7799 (comment)
Do I need to replicate this thing for the case when a job is added (registered) too? Asking this because I couldn't understand if it was even necessary as I couldn't think of a way to reproduce that situation.

[code-sparsh]

Added the test case for it.

Also, are there no test cases present (or needed) for the Automation GUI?

[thc202]

Do I need to replicate this thing for the case when a job is added (registered) too?

Yes, that happens when the add-on is updated, rather than just being uninstalled. That case should still work, the plan will still have the job but the new version. (Note that we should not be changing the contents of the plan.)

There are no GUI tests currently.

[code-sparsh]

Yes, that happens when the add-on is updated, rather than just being uninstalled. That case should still work, the plan will still have the job but the new version. (Note that we should not be changing the contents of the plan.)

okay okay, got it.
Can you help me in re producing this situation? I mean how to push updates to an addOn in local dev environment?

is it related to copyZapAddOn command?

[thc202]

You can install/update the add-ons with the task installZapAddOn (e.g. :aO:reports:iZAO, assuming no API key and listening on default port).

[code-sparsh]

One problem I am facing is that, whenever the add-on is updated using the installZapAddOn task, it first uninstalls it (un-registers from Automation) and then installs it again with the new version. Now since the job is already removed from the plan upon uninstallation (based on my current changes), I cant access it in the registerAutomationJob method. How should I proceed with this?
I need to access it so that I add it again with the properties and settings it had before.

[code-sparsh]

Update: Still not able to figure out a way to handle that problem.

The first part (add-on uninstallation case) is working fine though. But Since the update process goes like this only (first uninstall then re install), I am not sure if it's even possible to do the second part.

Currently, with these changes an add-on update will remove that job from the plan just like an un-installation instead of retaining that job in the plan with the newer version. But still, it would atleast be able to prevent those exceptions thrown at run-time.

Please let me know your views on this... as I was also wondering if I should just go ahead with the current changes and make it un-draft? Or should I wait and keep the PR the way it is, for now.
@kingthorin @thc202

[ArkaprabhaChakraborty]

How about adding the necessary check? If it is present or not? If it is present the default action of uninstall and reinstall fits! if it's not then just installing the latest should be the way to go.