FLIP - Extended Transaction Format

[JeffreyDoyle]

No description provided.

[dete]

I'm very excited by this proposal! I think this shows a path towards a much better way of letting apps and wallets "negotiate" on what responsibility each of them has to provide loads of flexibility to app developers, without compromising on security.

Some thoughts:

• What if all variables for each Role were kept together? I think this would make things much more readable. (In particular, you could enforce having a single #Role tag for each distinct role, instead of spreading them throughout the file. This will increase auditability, I believe.)
• I think asking Cadence to enforce that specific prepare() statements update specific variables might be "breaking the wall" between core language vs templating considerations. I agree that good coding practice and FLIX might require this, but asking the language to enforce it is asking the language to enforce syntax on comments. We should either promote Roles to a language feature, or let the language ignore them. My instinct is the latter (which makes it so the templating rules can change without language changes), in which case the only thing the language would enforce is that all transaction variables are initialized exactly once between all prepare() blocks.
• Weirdly, I don't like including empty prepare() statements in the template. I can't defend this rationally. 😅
• I think the example would be more compelling if you presented both a complete template (as it would be included in FLIX), the code that the app would present (which would have execute() filled in) and then be specific about the additions made by both the buyers and sellers wallets.
• I think both the purchase price and the NFT ID should be arguments to the tx. This will make it easier for both the wallet and the application to make sure they are referencing the same thing.
• Overall, I think fleshing out the example more is going to show some rough edges to this proposal. For example, your comment says "check that Buyer received the NFT", but how? What if the buyer doesn't have a public capability on their collection that allows arbitrary code to query ownership? Does the post condition have access to the authAccounts?

[bluesign]

This is a great idea, I was long time defending this approach.

I few comments:

• Multisign, as an example, makes it a bit more complicated than it is. I think we should open up with simple single-signer transactions and then build on that slowly. I think the best entry-level example would be sending someone NFT.

• pre/post-conditions are nice to have, but I am not sure if they are enough. I am still in favor of simulating the transaction and auto-fill the post conditions.

• Some multi-step transaction examples also can shine here, something like exchanging FLOW to USDC, with this USDC, buying a kitty NFT, send this NFT to a third party. Instead of focusing on multiple actors, I think we should focus on multiple steps.

• This is a bit bonus point, but I believe also we should utilize Type a lot more in templated transactions. ( can be in pre/post conditions or directly in the fields )

• I also feel this ( the fields in the transaction ) will bloat a lot when we have multi-steps

[katelynsills]

Very excited about this! I’ve been thinking a lot about the problem of transactions assuming the user's storage.

The problem is that no single party knows how to create the transactions, and the user, wallet, and app have to coordinate:

• Only the user knows their preferences
• Only the wallet knows where the user has stored things and how to retrieve them (assuming a non-power user)
• Only the app knows the domain-specific UI and business logic (assuming wallets are general purpose and aren’t tailored to apps)

The coordination is further complicated by the fact that the application could be malicious.

Re: viable alternatives, I’m wondering if the app actually needs the information prior to the transaction, not in the transaction. This is because only the app has the domain-specific UI which can provide the user a rich environment. I’m thinking of this like a permission request mechanism between the app and the wallet. For example, if as a user, I’m trying to build my fighting robot from NFT parts, I actually want to use the app UI, not my wallet UI, to put the robot together. The wallet doesn’t know anything about robots and how they go together, let alone how to display it, so the application needs to make a request to the wallet to get a list of all the user’s robot parts. It’s only after the user has built their robot (and possibly selected new parts from the marketplace to buy) that the app would submit a transaction.

Essentially, once you factor in the need for the user to make choices through the application's UI, then the user's wallet is the wrong entity to be filling in a template. The user is making those choices outside the wallet.

But then there’s a remaining question of how to ensure that the end-result transaction actually matches the choices that the user/wallet made, and isn’t deceptively trying to steal assets. “Have the user read the transaction and approve or disapprove” is the fallback because it’s the easiest, but it’s also the worst for user protection. I thought Dete’s question about whether the post statement has AuthAccount access was interesting. Could it be something scoped to this application instead, for POLA reasons? Maybe the transaction gets flagged as probably malicious if the transaction uses anything beyond what was already requested by the application?

[turbolent]

Nice proposal!

[turbolent]

Cadence already has support for pragma declarations, which have the syntax '#' expression. They are not used for anything yet, but were added early on in anticipation of such features. A use like #role: Seller is thus currently syntactically invalid. A valid use could be e.g. #role(Seller), which in terms of the AST would be expressed as a invocation expression of identifier role, with one argument, the identifier Seller. Another valid use could be e.g. #tx(role: Seller).

Also, pragma declarations are self-standing, they are not associated with any other declaration. However, for a transaction declaration, Cadence could look specifically for preceding pragma declarations and consider them.

[bjartek]

Another alternative here is to parse the docstring of said transaction, i have a very crude attempt at that in a branch in overflow. the transaction i process there is here https://github.com/bjartek/overflow/blob/flix2/transactions/mint_tokens.cdc

However i think pragma is the better approach here, especially if it can be syntax checked. Since a comment is well a comment.

[SupunS]

Each prepare statment would have the ability to initialize transaction varaibles, but not read their values. Each prepare statment can only assign variables annotated with the same role as itself.

This sounds more like scoping to me. What if a new "role block" is introduced?
Something like:

role seller {
    let nft: @NonFungibleToken.NFT

    prepare()
}

role buyer {
    let receiver: Capability<&{NFT.Receiver}>

    prepare()
}

So that it's clear on the scoping rules, and also:

• Don't have to repeat the pragma for every variable/prepare block
• Has less chance of making mistakes. e.g: assigning the wrong role (wrong pragma) to the wrong variable.

I'm not quite sure what the variable access semantic should be though. One option is to make them seamlessly available to execute and post blocks. Another option would be to use role-qualified-names. e.g: seller.nft, etc. The latter would allow you to have separate namespaces for roles (so don't have to worry about duplicate names)

[turbolent]

Great idea! This scoped approach would also solve the problem of definite initialization in the prepare block 👍

[dsainati1]

I really like this idea! Having a new block would solve this and the related problems pretty neatly, and also be fairly clear to a reader.

I'm curious though whether pre and post blocks need to be similarly separated though; as with the upcoming changes in Stable Cadence to restrict condition blocks to being view, they won't be subject to the same concerns about malicious use of accounts and data that prepare functions are. Seems reasonable to have these conditions exist outside of role blocks and make any variables defined within those blocks available outside them.

[turbolent]

While implementing type checking for role blocks I realized the current resource semantics will not allow roles blocks to have resource-kinded fields, because they somehow need to be invalidated.

In the case of a transaction, the execute block acts as the destructor and allows invalidation of resource-kinded fields.

However, in role blocks, there is no execute block, but rather the outer transaction's execute block should probably be allowed to invalidate role blocks' resource kinded fields.

For example, we probably want to allow something like this example, which is currently rejected (correctly) due to the nested move in the last statement.

resource R {}

fun absorb(_ r: @R) {
    destroy r
}

transaction {

    role buyer {
        var x: @R

        prepare() {
            self.x <- create R()
        }
    }

    execute {
        absorb(<-self.buyer.x)
    }
}

Trying to work around this by e.g. making the field optional, for which nested moves are allowed, still gets rejected (correctly), because the optional resource-kinded field could still have a value, and there is no destructor which invalidates it.

resource R {}

fun absorb(_ r: @R) {
    destroy r
}

transaction {
    role buyer {
        var x: @R?

        prepare() {
            self.x <- create R()
        }

    }

    execute {
        let x <- self.buyer.x <- nil
        absorb(<-x!)
    }
}

Support for resource-kinded fields in roles is probably wanted, for example one could imagine a buyer role which gets the FT Vault needed to purchase an item.

If we do want to support resource-kinded fields in roles, should we extend resource invalidation and allow for nested field invalidation? For the first example above: the transaction's buyer field is indirectly resource-kinded, because it has a resource-kinded field, so must be invalidated – given that roles are not first-class values, at the role's resource-kinded fields must be invalidated. Currently this is limited to just fields of self.

[dsainati1]

What if instead of thinking of the role like an actual object in the language with fields of its own, we treated it like a namespace? This way we wouldn't have to special case its "fields", because those fields would exist on the outer transaction's scope, they would just be namespaced to their role.

[SupunS]

What if instead of thinking of the role like an actual object in the language with fields of its own, we treated it like a namespace?

I like this idea. This is exactly how I had imagined when I first proposed the syntax. Role-block is just a named-block/named-scope and nothing else. For eg., many languages support the block syntax { } at the statement level to allow having a separate scope for certain things, and this is also that, but with a label.

[turbolent]

How would this namespacing be implemented? I'm not saying it couldn't, I'm really just trying to think through this alternative and how it could be achieved, as well as how it would compare to other solutions.

For the definite initialization checking in the prepare blocks, the transaction's prepare block would need to ignore all of the roles' fields, and vice-versa, each role block would need to only consider its own fields and ignore all other fields?

For the resource checking, we would still need to adjust it and allow nested invalidation, because AFAICS currently it is implemented on a syntax level. For example, in the example above, <-self.buyer.x would internally just refer to self's "buyer_x" field, but syntactically it still looks like a nested access.

[turbolent]

Luckily, allowing nested invalidation of resource-kinded fields of transaction roles turned out to be not hard to implement: onflow/cadence#2262. I think that resolves this discussion

[turbolent]

Supporting multiple prepare blocks, post conditions, etc. in the parser is no problem.

Given that prepare blocks currently checked like initializers, i.e. they must initialize all fields of the transaction, this check would need to be extended to support partial initialization in one block, and only consider all blocks together as the total set of initialized fields.

[turbolent]

Do I understand it correctly, that the idea is that this empty template is presented to multiple wallets, and they each fill in this empty/partially filled transaction?
In that case we would need to expand the syntax to allow empty blocks.

[bjartek]

From prior discussions on this I belive this assumption is true.

[bluesign]

It will be ping pong little, proposer will propose, then it will go wallets they will add their prepares, it will go back to proposer, then proposer will merge them, then it will go to signers, each of them will sign and send back to proposer, then proposer will send to payer, payer will sign, then send back to proposer, proposer will send to network :)

[turbolent]

How would a case of multiple prepare statements with varying prepare block parameter lists be handled?

[bjartek]

I am not OP here, but we have discussed this a bit in various meetings and I am quite passionate about this so i I will chime in. Feel free to correct me @JeffreyDoyle.

My understanding is that each prepare block can be filled out indepedently and composed into a transaction.
So lets say we have a buyer and a seller with two prepare blocks, for the seller we need to prepare the receiving vault while for the buyer we might prepare both the sending vault and the receiving collection/capability.

This also ties into flix where each flix can be a single role here if my understanding is correct. So the buyer pre/post pair would be one flix and the seller pre/post could be another.

[bjartek]

From prior discussions on this I belive this assumption is true.

[bjartek]

I am not OP here, but we have discussed this a bit in various meetings and I am quite passionate about this so i I will chime in. Feel free to correct me @JeffreyDoyle.

My understanding is that each prepare block can be filled out indepedently and composed into a transaction.
So lets say we have a buyer and a seller with two prepare blocks, for the seller we need to prepare the receiving vault while for the buyer we might prepare both the sending vault and the receiving collection/capability.

This also ties into flix where each flix can be a single role here if my understanding is correct. So the buyer pre/post pair would be one flix and the seller pre/post could be another.

[bjartek]

And that the seller has 20 flow less. @bluesign have propossed that mutating state should have mandatory post conditions. It is a lot of work but it is very safe.

[bjartek]

What about royalty in this scenario? Should the transaction care about that aswell. In .find for instance royalty is handled by the contract not by the transaction, so that it cannot be skipped. In this case i guess the pre block for this role would fetch multiple variables for fund receivers and assert on all of them.

[SupunS]

Each prepare statment would have the ability to initialize transaction varaibles, but not read their values. Each prepare statment can only assign variables annotated with the same role as itself.

This sounds more like scoping to me. What if a new "role block" is introduced?
Something like:

role seller {
    let nft: @NonFungibleToken.NFT

    prepare()
}

role buyer {
    let receiver: Capability<&{NFT.Receiver}>

    prepare()
}

So that it's clear on the scoping rules, and also:

• Don't have to repeat the pragma for every variable/prepare block
• Has less chance of making mistakes. e.g: assigning the wrong role (wrong pragma) to the wrong variable.

I'm not quite sure what the variable access semantic should be though. One option is to make them seamlessly available to execute and post blocks. Another option would be to use role-qualified-names. e.g: seller.nft, etc. The latter would allow you to have separate namespaces for roles (so don't have to worry about duplicate names)

[SupunS]

Can all post-statements access variables with any role?

I assume the execute statement can access any variable regardless of the role? Can they mutate though?

[turbolent]

Implemented support for nested pragma declarations in Cadence in onflow/cadence#2169

[bluesign]

My only concern is; there is need for cadence parser everywhere. I really like the pragmas as native language feature, though I am also almost sure people will use regex instead. So maybe we need very light cadence parser ( native js possibly )

[bluesign]

I am thinking a bit different again ( in the context of flix and this FLIP, synergy between them )

I think we can define interactions ( like in flix ) in smaller pieces where we can assign them to roles. For me it is a bit hard to describe in words but I will try my best.

What if we have some units ( I will call them actions ) that make state modification, and those can be assigned to roles.

For example: ( in the NFT vs FT trade example in the FLIP )

Buyer:

• gives FT ( FlowToken ( 20.0 ) )
• gets NFT ( Deniz.NFT ( 42 ) )

Seller:

• gives NFT
• gets FT

Here if we had : 4 flix action ( giveNFT, giveFT, getNFT, getFT ) and all would have it's own pre/post/prepare/execute statements.

We could have nice UX:

Buyer:

• gives FT ( FlowToken ( 20.0 ) ) -> - Your FlowToken Balance will decrease by 20.0
• gets NFT ( Deniz.NFT ( 42 ) ) ->.- Your Deniz.NFT collection will add NFT ( 42 ) [ probably metadataviews can be useful here ]

Seller:

• gives NFT -> Your Deniz.NFT collection will remove NFT ( 42 )
• gets FT -> Your flowToken balance will increase by 20.0

As those will be some kind of standard actions, technically we can make them struct. Implementing some generic interface. Possibly running their transactions pre/post/prepare/execute separately. Wallets would just need to create structs, possibly we would not even need separate prepare statements.

Possibly we can have a generic transaction runner transaction. ( considering we can assume passing auth account to flix would be safe, as they will be pinned by hash )

PS: I am just thinking this as a subset of this FLIP, something that would be nice to support alongside

[nvdtf]

This is great. Looking forward to a cleaner separation of concerns between dapp and wallet enabled by the improvements discussed here.

I was wondering if we could also enable the wallet to provide functionality (functions) rather than variables only. Let's see how well I can explain:

Problem
I'm assuming a [secondary] goal of this FLIP is for the wallets to produce secure and correct messages like "This transaction will do X with your Y asset" for any supported transaction. Secure interaction is enabled by exposing limited variables from the user's account using directives in the transaction. The problem is those variables are manipulated outside of the wallet's generated code, so the wallet needs to rely on post statements to make sure the exposed variables are not abused and the intended action is executed correctly.

Let's introduce an example: An NFT game has provided you with an asset of type T that you can upgrade by calling upgrade(). With the current approach, the developer can declare a variable of type @T that is initialized by the Player role. The only way for the wallet to make sure only upgrade is called in a given transaction is to verify somehow in a post statement by checking all the other stuff that can go wrong. This will introduce mandatory "verifiability" property to all mutations.

Solution
A basic idea to solve this is to upgrade role blocks into interfaces so the wallet can produce wrappers as well. Interfaces in Cadence can dictate what variables or functions are required without providing a concrete implementation. In this approach, the new transaction format will enable prepare to accept interfaces that are defined in the transaction CDC file. Wallets can provide concrete implementations for those interfaces in separate code blocks. Developers can also mock the interfaces if needed during testing.

For the previous example the wallet can provide the implementation for upgrade(T) that will make sure no other functions are called on the exposed capability.

Sorry if this is too half-baked, wanted to know what you think. I'm sure there are blind spots that I might have missed here.

[joshuahannan]

Maybe I just need more time to think about it, but I don't see how this can be that beneficial besides to a very small subset of power users. I imagine that 99% of users will store their assets in standard paths and use standard interactions, so my preference is to try to make those kind of interactions as simple as possible for wallets and users and try to move as much logic as possible into contracts, which can be audited better. I just looked at this for the first time today though, so I'll need some more time to ponder it.

[katelynsills]

Would it be possible to write out the basic use case for this? I think I'm still trying to grasp exactly what kind of things the wallet (and not the application) might be able to fill in. If I understand correctly, this FLIP is limited to what the wallet specifically can fill in, right?

During the meeting, it sounded like there were two kinds of "slots":

1. A wildcard, application-determined execute statement. Nothing in the execute statement can hurt the account, if we pin down the transfer of an NFT for 20 tokens.
2. The transaction arguments - variables that the transaction template is aware of. E.g. buying NFT with ID X for Y money.

Even for the basic use case, isn't the user still probably interacting through the application UI, so shouldn't that be filling in the slots instead of the wallet? For example, if I buy an NFT in a marketplace, I usually click on the NFT in the application UI, not through my wallet.

I'm guessing one or more of my assumptions here are wrong, but I'm still trying to pin down which. :)

[bjartek]

From my limited understanding @katelynsills in that above example what the wallet can fill out is

• where to put the nft you bought
• what vault to pay with

[katelynsills]

Thanks @bjartek!

It sounds like the wallet is limited to handling the "checkout" process then. Just like normal retail requires you to choose which credit card you want to use and what address to send it to, in this case it's which vault to use to pay and which collection to put the NFT in. This would mean that the application handles everything else, including the storefront and selection of what to purchase and for how much.

If this sounds right, I think it would help to identify the "checkout" use case as the primary use case for this FLIP, so that it's clear what we actually expect the wallet to be able to fill in.

[JeffreyDoyle]

Hi all, we have some updates to share!


In this FLIPs working group we have decided on the following:

• Adopt the Role Block format as proposed by @SupunS https://github.com/onflow/flips/pull/41#discussion_r1032669506


We have also began to think about & discuss:



• Ways to minimize the complexity involved for wallets to codegen their assigned prepare statements. 

  • Some ideas that have been proposed include:

    • Enumerating some common request types that could be given to wallets so they could, for example, respond with where a given type should go to or come from in a given account, or some cadence code to assign a variable. For example, given a type, a client could ask a wallet where that type should be stored in their account (eg: path to a collection that holds that type).
    • Client libraries could inspect a cadence transaction to see if it contains these requests, and execute them against the wallet associated with the role that request lives within:


role seller { 
    let nft: @NonFungibleToken.NFT 
    prepare() { 

      let x: StoragePath = WALLET.STORAGEPATH(@NonFungibleToken.NFT) | "/storage/foo" // Fallback to /storage/foo if wallet does not support this feature.
    
      ... 

    } 

} 




role buyer { 


  let receiver: Capability<&{NFT.Receiver}> 
  

  let vault: @FlowToken.Vault 
  

  prepare() { 
   

    let x: StoragePath = WALLET.STORAGEPATH({ type: @NonFungibleToken.NFT }) | “/storage/foo” // Fallback to /storage/foo if wallet does not support this feature.


    ... 
    

    vault = WALLET.VAULT({ type: @FlowToken.Vault, amount: "20.0" }) | … 
cadence code to assign vault... // Fallback to the cadence code provided if wallet does not support this feature.


   } 
}


• • • The way the request could be exchanged between a client and wallet could be established as an FCL service, which FCL libraries could use.

    • The benefit with this approach being that it potentially lessens the complexity of the codegen process by requiring wallets to support responding to a known set of possible requests.

---

If you are not included in this FLIPs working group, and would like to be included, please message me on Discord (JeffD#6865) and we will add you to any future invites!

[dsainati1]

Wanted to clarify one thing while this is being implemented: Was the intention for each role block to have one signer, and that each signer be assigned to one role? Or is it okay for the prepare functions in each role to have the full list of signers in each case?

[bluesign]

Or is it okay for the prepare functions in each role to have the full list of signers in each case?

Roles should isolate signers, it would be no sense to have full list of signers in each case.

[JeffreyDoyle]

New update! Feb 24 2023

Hello everyone - we have some new updates to share!

Proposed Additions

As discussed in the previous breakout session, we're proposing adding a new concept to this FLIP known as the transaction resolve phase.

The transaction resolve phase would be responsible for dealing with the 'outputs' of a transaction. It functions much in a similar way as the prepare phase, which is responsible for providing the 'inputs' of a transaction.

The resolve phase would be a dedicated phase where resources can be stored after the execution phase, similar to how the prepare phase is where resources are retrieved before the execution phase.

The various phases of a transaction's execution would be: prepare -> pre -> execute -> resolve -> post

What's the benefit?

In this FLIP, wallets could produce the content of prepare and resolve phases of a transaction for their assigned role block. This enables transaction cadence developers to not have to concern themselves about how to retrieve and store resources, or otherwise engage with any wallet controlled accounts involved in the transaction. A transaction cadence developer would only need to be concerned with the roles of a transaction, what variables are involved in a transaction, and how they want to operate on those variables (the execution phase), and any pre/post statements they require.

💡 Key Idea: This pattern isolates the concern of the inputs/outputs of the transaction to the wallet, and the transaction logic to the cadence transaction developer.

A transaction with this new resolve phase in this FLIP might look something like:

transaction(nftId: UFix64, amount: UFix64) {

  var myExampleVar: Int
	
  role Buyer {
    input var payment: @FlowVault
    output var newNft: @NFT
	
    prepare(a: AuthAccount) {
      payment <- a.borrow<FlowVault>(/private/FlowVault)!.withdraw(amount)
	
      myExampleVar = 1
    }
    post {
      newNft.id == nftId: "NFT ID must be correct! 😤"
    }
    resolve(b: AuthAccount) {
      b.borrow<NFTCollection>(/private/NFTCollection)!.deposit(<-newNft)
    }
  }
	
  role Seller {
    input var newNft: @NFT
    output var payment: @FlowVault
	
    prepare(c: AuthAccount) {
      newNft <- c.borrow<NFTCollection>(/private/NFTCollection)!.withdraw(id: nftId)
    }
    post {
      payment.balance == amount: "Payment must be correct! 💰"
    }
    resolve(d: AuthAccount) {
      d.borrow<FlowVault>(/private/FlowVault)!.deposit(<-payment)
    }
  }
	
  execute {
    ...do stuff...
  }
}

Role blocks would include:

• Role specific transaction variables prefixed with 'input' or 'output'
  • The same variable name can be shared between role blocks, but can only once be declared input or output
  • These variables would all be accessible to the execute phase, but not other role blocks.
• Role specific prepare and resolve phases
  • The prepare phase(s) would be responsible for assigning a value to each 'input' variable in the role block
  • The resolve phase(s) would be responsible for storing each 'output' variable in the role block
• Any pre/post conditions the wallet requires

---

Want to join future breakout sessions?

If you are not included in this FLIPs working group, and would like to be included, please message me on Discord (JeffD#6865) and we will add you to any future invites!

[bluesign]

I think 99% of the transactions are single signer ( single Role, ignoring duc etc ), thats why I think this level of detail seems not enough for me.

I think instead of Roles, we can use on-chain constructed Actions ( like we had standard in metadata views ) an Action can have; order, inputs / outputs, pre/post and prepare/resolve phases. I will try to make an example poc contract for this till our next meeting.

[KshitijChaudhary666]

Hey @JeffreyDoyle - is there any update or movement on this?