[main-2.x][WIP] IDE Ledger support for upgrades

[moisesackerman-da]

No description provided.

[moisesackerman-da]

As expected, quite a few of the ported tests fail. Some of them due to using unvetDar (unsupported on the IDE ledger, so we can remove them (or put them behind a CPP pragma)), but the rest point to missing changes.

[moisesackerman-da]

these are the current test failures (note that tests : [(Text, Script ())] -> Script () only shows the first failure)

Fails if the signatories set gets smaller 
    Exception in thread "main" com.daml.lf.engine.free.InterpretationError: Error: Unhandled Daml exception: DA.Exception.AssertionFailed:AssertionFailed@3f4deaf1{ message = "Expected Upgrade error but got Right ()" }	at com.daml.lf.engine.script.v2.ScriptF$Throw.execute(ScriptF.scala:121)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner(ScriptF.scala:51)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner$(ScriptF.scala:47)	at com.daml.lf.engine.script.v2.ScriptF$Throw.executeWithRunner(ScriptF.scala:114)	at com.daml.lf.engine.script.v2.Runner.$anonfun$run$1(Runner.scala:99)	at com.daml.lf.engine.free.Free$Result.loop$2(Free.scala:72)	at com.daml.lf.engine.free.Free$Result.$anonfun$runF$1(Free.scala:72)	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)	at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110)	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110)	at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59)	at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57)	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)[INFO] [05/22/2024 15:03:05.808] [RunnerMain-pekko.actor.default-dispatcher-18] [CoordinatedShutdown(pekko://RunnerMain)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] (RunnerTestBase.scala:75)�[0m
Downgrade a contract with Nones when querying 
    Exception in thread "main" com.daml.lf.engine.script.Script$FailedCmd: Command QueryContractId failed: Failed to translate create argument: TypeMismatch(TTyCon(3f150b9b435f350124011b46518a016ae9bcc26dff8825f586e669f8dc8239f4:Query:QueryTemplate),ValueRecord(Some(83bb6c7b73b35aaeb6cad1fc5efb6b9e185780b8a960428b80b522b4105e3951:Query:QueryTemplate),ImmArray((Some(party),ValueParty(alice)),(Some(newField),ValueOptional(None)))),Mismatching variant id, the type tells us 3f150b9b435f350124011b46518a016ae9bcc26dff8825f586e669f8dc8239f4:Query:QueryTemplate, but the value tells us 83bb6c7b73b35aaeb6cad1fc5efb6b9e185780b8a960428b80b522b4105e3951:Query:QueryTemplate)Daml stacktrace:queryContractId at 80f84dc6f6008eac20e904d2714df6d66ce6e43680b38ef69245d0d0c2be194c:Query:49	at com.daml.lf.engine.script.v2.Runner.$anonfun$remapQ$2(Runner.scala:76)	at com.daml.lf.engine.free.Free$Result$Ask.$anonfun$transform$1(Free.scala:99)	at com.daml.lf.engine.free.Free$Result$Final.transform(Free.scala:87)	at com.daml.lf.engine.free.Free$Result.$anonfun$runF$1(Free.scala:72)	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)	at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110)	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110)	at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59)	at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57)	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)Caused by: com.daml.script.converter.ConverterException: Failed to translate create argument: TypeMismatch(TTyCon(3f150b9b435f350124011b46518a016ae9bcc26dff8825f586e669f8dc8239f4:Query:QueryTemplate),ValueRecord(Some(83bb6c7b73b35aaeb6cad1fc5efb6b9e185780b8a960428b80b522b4105e3951:Query:QueryTemplate),ImmArray((Some(party),ValueParty(alice)),(Some(newField),ValueOptional(None)))),Mismatching variant id, the type tells us 3f150b9b435f350124011b46518a016ae9bcc26dff8825f586e669f8dc8239f4:Query:QueryTemplate, but the value tells us 83bb6c7b73b35aaeb6cad1fc5efb6b9e185780b8a960428b80b522b4105e3951:Query:QueryTemplate)	at com.daml.lf.engine.script.ConverterMethods.toFuture(Converter.scala:129)	at com.daml.lf.engine.script.v2.ScriptF$QueryContractId.$anonfun$execute$8(ScriptF.scala:298)	... 13 more[INFO] [05/22/2024 15:03:38.739] [RunnerMain-pekko.actor.default-dispatcher-10] [CoordinatedShutdown(pekko://RunnerMain)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] (RunnerTestBase.scala:75)�[0m
Upgrade fails if variant has fields removed 
    Exception in thread "main" com.daml.lf.engine.free.InterpretationError: Error: Unhandled Daml exception: DA.Exception.AssertionFailed:AssertionFailed@3f4deaf1{ message = "Expected specific failure but got Left Unknown error: com.daml.lf.speedy.SError$SErrorCrash: CRASH (com.daml.lf.speedy.Speedy.Machine.go): Unexpected non-optional extra contract field encountered during downgrading." }	at com.daml.lf.engine.script.v2.ScriptF$Throw.execute(ScriptF.scala:121)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner(ScriptF.scala:51)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner$(ScriptF.scala:47)	at com.daml.lf.engine.script.v2.ScriptF$Throw.executeWithRunner(ScriptF.scala:114)	at com.daml.lf.engine.script.v2.Runner.$anonfun$run$1(Runner.scala:99)	at com.daml.lf.engine.free.Free$Result.loop$2(Free.scala:72)	at com.daml.lf.engine.free.Free$Result.$anonfun$runF$1(Free.scala:72)	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)	at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110)	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110)	at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59)	at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57)	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)[INFO] [05/22/2024 15:03:51.842] [RunnerMain-pekko.actor.default-dispatcher-5] [CoordinatedShutdown(pekko://RunnerMain)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] (RunnerTestBase.scala:75)�[0m
Fails if fields are removed 
    Exception in thread "main" com.daml.lf.engine.free.InterpretationError: Error: Unhandled Daml exception: DA.Exception.AssertionFailed:AssertionFailed@3f4deaf1{ message = "Expected specific failure but got Left Unknown error: com.daml.lf.speedy.SError$SErrorCrash: CRASH (com.daml.lf.speedy.Speedy.Machine.go): Unexpected non-optional extra contract field encountered during downgrading." }	at com.daml.lf.engine.script.v2.ScriptF$Throw.execute(ScriptF.scala:121)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner(ScriptF.scala:51)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner$(ScriptF.scala:47)	at com.daml.lf.engine.script.v2.ScriptF$Throw.executeWithRunner(ScriptF.scala:114)	at com.daml.lf.engine.script.v2.Runner.$anonfun$run$1(Runner.scala:99)	at com.daml.lf.engine.free.Free$Result.loop$2(Free.scala:72)	at com.daml.lf.engine.free.Free$Result.$anonfun$runF$1(Free.scala:72)	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)	at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110)	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110)	at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59)	at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57)	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)[INFO] [05/22/2024 15:04:26.237] [RunnerMain-pekko.actor.default-dispatcher-8] [CoordinatedShutdown(pekko://RunnerMain)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] (RunnerTestBase.scala:75)�[0m
Fails implicitly calling a removed V1 choice on a V2 contract, as V2 is selected 
    Exception in thread "main" com.daml.lf.engine.free.InterpretationError: Error: Unhandled Daml exception: DA.Exception.AssertionFailed:AssertionFailed@3f4deaf1{ message = "Expected unknown choice error, got Left Unknown error: Lookup error: NotFound(TemplateChoice(2a6c12b2284fe8817efc3ab047640b1a66df909dc5c9cd58bf016c90d1eb3494:AddedRemovedChoice:AddedRemovedChoiceTemplate,OldRemovedChoice),TemplateChoice(2a6c12b2284fe8817efc3ab047640b1a66df909dc5c9cd58bf016c90d1eb3494:AddedRemovedChoice:AddedRemovedChoiceTemplate,OldRemovedChoice))" }	at com.daml.lf.engine.script.v2.ScriptF$Throw.execute(ScriptF.scala:121)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner(ScriptF.scala:51)	at com.daml.lf.engine.script.v2.ScriptF$Cmd.executeWithRunner$(ScriptF.scala:47)	at com.daml.lf.engine.script.v2.ScriptF$Throw.executeWithRunner(ScriptF.scala:114)	at com.daml.lf.engine.script.v2.Runner.$anonfun$run$1(Runner.scala:99)	at com.daml.lf.engine.free.Free$Result.loop$2(Free.scala:72)	at com.daml.lf.engine.free.Free$Result.$anonfun$runF$1(Free.scala:72)	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)	at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110)	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110)	at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59)	at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57)	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)[INFO] [05/22/2024 15:04:50.844] [RunnerMain-pekko.actor.default-dispatcher-13] [CoordinatedShutdown(pekko://RunnerMain)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] (RunnerTestBase.scala:75)�[0m
Submitting participant has v2, other has v1, expect v1 used 
    Exception in thread "main" com.daml.lf.engine.script.Script$FailedCmd: Command UnvetDar failed: unvetDar is not supported when running Daml Script over the script serviceDaml stacktrace:unvetDarOnParticipant at f69517f3879db3e6ef27ed0feaea94cbaa2e98b2ab59f685eb9e0d0d7f50e986:UpgradeTestLib:41	at com.daml.lf.engine.script.v2.Runner.$anonfun$remapQ$2(Runner.scala:76)	at com.daml.lf.engine.free.Free$Result$Ask.$anonfun$transform$1(Free.scala:99)	at com.daml.lf.engine.free.Free$Result$Final.transform(Free.scala:87)	at com.daml.lf.engine.free.Free$Result.$anonfun$runF$1(Free.scala:72)	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:470)	at org.apache.pekko.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:73)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:110)	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:94)	at org.apache.pekko.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:110)	at org.apache.pekko.dispatch.TaskInvocation.run(AbstractDispatcher.scala:59)	at org.apache.pekko.dispatch.ForkJoinExecutorConfigurator$PekkoForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:57)	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)Caused by: java.lang.UnsupportedOperationException: unvetDar is not supported when running Daml Script over the script service	at com.daml.lf.engine.script.v2.ledgerinteraction.ScriptLedgerClient.unsupportedOn(ScriptLedgerClient.scala:130)	at com.daml.lf.engine.script.v2.ledgerinteraction.ScriptLedgerClient.unsupportedOn$(ScriptLedgerClient.scala:127)	at com.daml.lf.engine.script.v2.ledgerinteraction.IdeLedgerClient.unsupportedOn(IdeLedgerClient.scala:49)	at com.daml.lf.engine.script.v2.ledgerinteraction.IdeLedgerClient.unvetDar(IdeLedgerClient.scala:970)	at com.daml.lf.engine.script.v2.ScriptF$UnvetDar.$anonfun$execute$88(ScriptF.scala:774)	... 13 more[INFO] [05/22/2024 15:05:13.655] [RunnerMain-pekko.actor.default-dispatcher-4] [CoordinatedShutdown(pekko://RunnerMain)] Running CoordinatedShutdown with reason [ActorSystemTerminateReason] (RunnerTestBase.scala:75)�[0m