Switching solr subsystem from the admin console fails because of duplicate quartz job name

[vierbergenlars]

Environment

• Alfred Telemetry 0.7.1
• Alfresco Enterprise 6.2.1

Steps to reproduce

1. Install Alfred Telemetry in a standard Alfresco Enterprise war
2. Start up Alfresco
3. Navigate to admin console, select Search Services and switch Search service in use to a different value. Click save

Expected behavior

1. Changes to search subsystem are saved and the selected subsystem is started. The other subsystem is shut down.
2. Metrics are now only collected from the new subsystem, gathering metrics from the old subsystem is shut down.

Actual behavior

Tomcat HTTP 400 error page, changes made are NOT saved.
This error page is due to a redirect by the admin page that contains the "message" in the URL.

The actual error, from the logs is:

2021-09-10 13:18:20,244  INFO  [management.subsystems.ChildApplicationContextFactory] [http-nio-8080-exec-4] Starting 'Search' subsystem, ID: [Search, managed, solr6]
2021-09-10 13:18:20,767  INFO  [binder.solr.SolrMetricsBeanPostProcessor] [http-nio-8080-exec-4] Registered SolrTrackingMetrics bean
2021-09-10 13:18:20,769  INFO  [binder.solr.SolrMetricsBeanPostProcessor] [http-nio-8080-exec-4] Registered SolrShardingMetricsFactory bean
2021-09-10 13:18:21,660  INFO  [solr.tracking.SolrTrackingMetrics] [http-nio-8080-exec-4] Registering Solr metrics
2021-09-10 13:18:21,679  INFO  [management.subsystems.ChildApplicationContextFactory] [http-nio-8080-exec-4] Startup of 'Search' subsystem, ID: [Search, managed, solr6] complete
Debugger failed to attach: handshake failed - connection prematurally closed
2021-09-10 13:19:33,267  INFO  [management.subsystems.ChildApplicationContextFactory] [http-nio-8080-exec-9] Starting 'Search' subsystem, ID: [Search, managed, noindex]
2021-09-10 13:19:33,351  INFO  [binder.solr.SolrMetricsBeanPostProcessor] [http-nio-8080-exec-9] Registered SolrTrackingMetrics bean
2021-09-10 13:19:33,352  INFO  [binder.solr.SolrMetricsBeanPostProcessor] [http-nio-8080-exec-9] Registered SolrShardingMetricsFactory bean
2021-09-10 13:19:33,386  INFO  [solr.tracking.SolrTrackingMetrics] [http-nio-8080-exec-9] Registering Solr metrics
2021-09-10 13:19:33,389  WARN  [management.subsystems.ChildApplicationContextFactory$ChildApplicationContext] [http-nio-8080-exec-9] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'eu.xenit.alfred.telemetry.binder.solr.sharding.SolrShardingMetrics': Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [eu.xenit.alfred.telemetry.binder.solr.sharding.SolrShardingMetricsFactory]: Constructor threw exception; nested exception is org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'DEFAULT.SolrShardingMetricsJob', because one already exists with this identification.
2021-09-10 13:19:33,391  WARN  [management.subsystems.ChildApplicationContextFactory] [http-nio-8080-exec-9] Startup of 'Search' subsystem, ID: [Search, managed, noindex] failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'eu.xenit.alfred.telemetry.binder.solr.sharding.SolrShardingMetrics': Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [eu.xenit.alfred.telemetry.binder.solr.sharding.SolrShardingMetricsFactory]: Constructor threw exception; nested exception is org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'DEFAULT.SolrShardingMetricsJob', because one already exists with this identification.
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:304)
	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:285)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1340)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1186)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:515)
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:320)
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226)
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:318)
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:860)
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:877)
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:549)
	at org.alfresco.repo.management.subsystems.ChildApplicationContextFactory$ApplicationContextState.start(ChildApplicationContextFactory.java:824)
	at org.alfresco.repo.management.subsystems.AbstractPropertyBackedBean.start(AbstractPropertyBackedBean.java:1098)
	at org.alfresco.repo.management.subsystems.AbstractPropertyBackedBean.setPropertiesInternal(AbstractPropertyBackedBean.java:748)
	at org.alfresco.repo.management.subsystems.AbstractPropertyBackedBean.setProperties(AbstractPropertyBackedBean.java:903)
	at org.alfresco.enterprise.repo.management.subsystems.PropertyBackedBeanAdapter$5.execute(PropertyBackedBeanAdapter.java:585)
	at org.alfresco.enterprise.repo.management.subsystems.PropertyBackedBeanAdapter$5.execute(PropertyBackedBeanAdapter.java:550)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.enterprise.repo.management.MBeanSupport$2.doWork(MBeanSupport.java:120)
	at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:602)
	at org.alfresco.enterprise.repo.management.MBeanSupport.doWork(MBeanSupport.java:108)
	at org.alfresco.enterprise.repo.management.subsystems.PropertyBackedBeanAdapter.setAttributes(PropertyBackedBeanAdapter.java:549)
	at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.setAttributes(DefaultMBeanServerInterceptor.java:792)
	at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.setAttributes(JmxMBeanServer.java:768)
	at org.alfresco.enterprise.repo.management.script.ScriptMBean.saveImpl(ScriptMBean.java:210)
	at org.alfresco.enterprise.repo.management.script.JmxScriptImpl.save(JmxScriptImpl.java:108)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
	at com.sun.proxy.$Proxy184.save(Unknown Source)
	at org.alfresco.enterprise.repo.management.script.JmxScriptProcessorExtension.save(JmxScriptProcessorExtension.java:45)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:138)
	at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:226)
	at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:45)
	at org.mozilla.javascript.gen.classpath__alfresco_enterprise_webscripts_org_alfresco_enterprise_repository_admin_repository_services_admin_searchservice_post_js_2._c_main_17(classpath*:alfresco/enterprise/webscripts/org/alfresco/enterprise/repository/admin/repository-services/admin-searchservice.post.js:509)
	at org.mozilla.javascript.gen.classpath__alfresco_enterprise_webscripts_org_alfresco_enterprise_repository_admin_repository_services_admin_searchservice_post_js_2.call(classpath*:alfresco/enterprise/webscripts/org/alfresco/enterprise/repository/admin/repository-services/admin-searchservice.post.js)
	at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:87)
	at org.mozilla.javascript.gen.classpath__alfresco_enterprise_webscripts_org_alfresco_enterprise_repository_admin_repository_services_admin_searchservice_post_js_2._c_script_0(classpath*:alfresco/enterprise/webscripts/org/alfresco/enterprise/repository/admin/repository-services/admin-searchservice.post.js:525)
	at org.mozilla.javascript.gen.classpath__alfresco_enterprise_webscripts_org_alfresco_enterprise_repository_admin_repository_services_admin_searchservice_post_js_2.call(classpath*:alfresco/enterprise/webscripts/org/alfresco/enterprise/repository/admin/repository-services/admin-searchservice.post.js)
	at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:412)
	at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3545)
	at org.mozilla.javascript.gen.classpath__alfresco_enterprise_webscripts_org_alfresco_enterprise_repository_admin_repository_services_admin_searchservice_post_js_2.call(classpath*:alfresco/enterprise/webscripts/org/alfresco/enterprise/repository/admin/repository-services/admin-searchservice.post.js)
	at org.mozilla.javascript.gen.classpath__alfresco_enterprise_webscripts_org_alfresco_enterprise_repository_admin_repository_services_admin_searchservice_post_js_2.exec(classpath*:alfresco/enterprise/webscripts/org/alfresco/enterprise/repository/admin/repository-services/admin-searchservice.post.js)
	at org.alfresco.repo.jscript.RhinoScriptProcessor.executeScriptImpl(RhinoScriptProcessor.java:513)
	at org.alfresco.repo.jscript.RhinoScriptProcessor.execute(RhinoScriptProcessor.java:210)
	at org.alfresco.repo.processor.ScriptServiceImpl.execute(ScriptServiceImpl.java:219)
	at org.alfresco.repo.processor.ScriptServiceImpl.executeScript(ScriptServiceImpl.java:181)
	at org.alfresco.repo.web.scripts.RepositoryScriptProcessor.executeScript(RepositoryScriptProcessor.java:109)
	at org.springframework.extensions.webscripts.AbstractWebScript.executeScript(AbstractWebScript.java:1376)
	at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:86)
	at org.alfresco.repo.web.scripts.RepositoryContainer$3.execute(RepositoryContainer.java:527)
	at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:595)
	at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:664)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScriptInternal(RepositoryContainer.java:435)
	at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:315)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:399)
	at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:210)
	at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:132)
	at org.alfresco.repo.web.scripts.AlfrescoWebScriptServlet.service(AlfrescoWebScriptServlet.java:43)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.alfresco.web.app.servlet.ServletMetricsFilter.doFilter(ServletMetricsFilter.java:161)
	at org.alfresco.repo.web.filter.beans.BeanProxyFilter.doFilter(BeanProxyFilter.java:89)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.extensions.webscripts.servlet.CSRFFilter.doFilter(CSRFFilter.java:349)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.alfresco.web.app.servlet.ClearSecurityContextFilter.doFilter(ClearSecurityContextFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:747)
	at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:616)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1626)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [eu.xenit.alfred.telemetry.binder.solr.sharding.SolrShardingMetricsFactory]: Constructor threw exception; nested exception is org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'DEFAULT.SolrShardingMetricsJob', because one already exists with this identification.
	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:184)
	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:300)
	... 109 more
Caused by: org.quartz.ObjectAlreadyExistsException: Unable to store Job : 'DEFAULT.SolrShardingMetricsJob', because one already exists with this identification.
	at org.quartz.simpl.RAMJobStore.storeJob(RAMJobStore.java:279)
	at org.quartz.simpl.RAMJobStore.storeJobAndTrigger(RAMJobStore.java:251)
	at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:855)
	at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249)
	at eu.xenit.alfred.telemetry.binder.solr.sharding.SolrShardingMetricsFactory.<init>(SolrShardingMetricsFactory.java:35)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:172)
	... 111 more