New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feature] #4225: Remove genesis signing #4382
base: main
Are you sure you want to change the base?
Conversation
4386499
to
5cb641f
Compare
tools/kagami/src/crypto.rs
Outdated
crypto_genesis_config.run(&mut genesis_buf_writer)?; | ||
|
||
let signed_genesis_from_file_encoded = fs::read(GEN_SIGNED_ENCODED_GENESIS_PATH)?; | ||
let _maybe_signed_genesis_from_file: GenesisNetwork = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be cool to assert_eq!
the resulting GenesisNetwork
with an expected one. Shouldn't take many lines for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've tried to make it work. It does not due to different creation times thus payload is different.
Do you have something in mind on how to bypass this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, that's unfortunate that there is this side effect sneaking deeply inside.
IMO this usage of current_time()
should be moved away from depths of data model, to some peripheral layers of code:
iroha/data_model/src/transaction.rs
Line 641 in 96bf997
let creation_time_ms = crate::current_time() |
Or add a way to mock what current_time
in tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or just compare by all fields except the creation_time_ms
tools/kagami/src/crypto.rs
Outdated
let key_pair = if let Some(path) = self.keypair_path { | ||
Self::read_keypair(path)? | ||
} else { | ||
let (public_key, private_key) = (self.get_public_key()?, self.get_private_key()?); | ||
KeyPair::new(public_key, private_key)? | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we always require KeyPair
maybe we can simplify code to only allow keypair?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought that it might be complex to pass KeyPair as a command line argument. And I thought that we don't want to enforce user to pass it through the file, so I decided to support both options
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree. Having both a --key-pair
and --private-key
/--public-key
is good for CLI, IMO.
Although, I just realized that --public-key
is redundant, as it might be derived from the private key. Maybe even having only --private-key
is enough.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Although, I just realized that --public-key is redundant, as it might be derived from the private key. Maybe even having only --private-key is enough.
it cannot always be derived depending on which algorithm is used
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree. Having both a --key-pair and --private-key/--public-key is good for CLI, IMO.
yes if they are mutually exclusive. But then again, this goes against your previous suggestion to keep things as flat as possible?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are mutually exclusive which is enforced by clap grouping. I think it's pretty convenient to have keypair in json and load it from the file. And the same goes for passing public and private key to cli
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is public key ever necessary when signing genesis?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see it's because of the fact that we pack public keys with signatures
ae4eda1
to
ace1afa
Compare
3ea17e9
to
7c963d7
Compare
genesis/Cargo.toml
Outdated
@@ -12,15 +12,17 @@ workspace = true | |||
|
|||
[dependencies] | |||
iroha_crypto = { workspace = true } | |||
iroha_data_model = { workspace = true, features = ["http"] } | |||
iroha_data_model = { workspace = true, features = ["http", "transparent_api"] } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As it was necessary to somehow setup a creation time I decided to make it public in the module with this feature. Maybe I'm missing some feature that would enable me to do just that without transparent_api
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is no need to enable transparent_api
here. You can use TransactionPayload::new_with_time
. This feature should only be enabled by iroha_core
@0x009922 I was actually thinking to rename this feature to _transparent_api
to give people the idea it's an internal feature and dissuade them from using it. What do you think about this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mversic, it would make sense, I think. And if it's really only for iroha_core
, naming it _transparent_api_for_iroha_core
would be clearer. I myself haven't understood the scope of it until now.
15b8b25
to
7feffe6
Compare
algorithm: crypto::AlgorithmArg, | ||
/// Private key (in string format) to sign genesis block | ||
#[clap(long, group = "private_key")] | ||
private_key_string: Option<String>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(in string format)
Which one? A JSON with algorithm
and payload
fields?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As there's a field for algorithm I thought of using hex string for private key and multihash for public. I've added hex
part
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean, SCALE-encoded hex?
It is not a usual format which users use with Iroha for private keys. Unfortunately, we always separate algorithm
and payload
somehow :(
This might help: #4412
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please replace String
with a type that shows what this string is, which shows how it parses it.
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
fix tests change test_env.py to reflect the changes Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
moved signing from kagami to genesis moved genesis signing from kagami_crypto to kagami_genesis some renames from PR Signed-off-by: VAmuzing <amuzik95@gmail.com>
add distinct error type for `try_from` for `SignedTransaction` change `GenesisNetwork` to have single transaction instead of a Vec some renamings Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
…ix some tests Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
…hange how data is passed to compact & full peers Signed-off-by: VAmuzing <amuzik95@gmail.com>
…cy.sh Signed-off-by: VAmuzing <amuzik95@gmail.com>
f1efc33
to
8e920f1
Compare
Signed-off-by: VAmuzing <amuzik95@gmail.com>
Signed-off-by: Vladislav <amuzik95@gmail.com>
Signed-off-by: VAmuzing <amuzik95@gmail.com>
5df5f9a
to
d15bda1
Compare
"--chain-id", | ||
chain_id_slice, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"--chain-id", | |
chain_id_slice, |
this should be in the genesis file, not given over CLI
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct me if I wrong, but I don't see genesis file containing chain id.
And it seems like every time we create genesis, be it inside the swarm or core libs, we supply chain id explicitly.
So where do we find it besides taking it over CLI?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct me if I wrong, but I don't see genesis file containing chain id.
Correct me if I wrong, but I don't see genesis file containing chain id.
And it seems like every time we create genesis, be it inside the swarm or core libs, we supply chain id explicitly.
So where do we find it besides taking it over CLI?
Yes, add it to the file
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Umm, duplicating the chain id in the genesis is unfortunate =(
Then we will have to decide what is the source of truth - Chain ID in the config file or in the genesis file? We will also have to validate their equality during config parsing.
So, maybe --chain-id
is fine? Another option might be to provide a relevant --config-file
, from where kagami
can read chain_id
.
"--genesis-file", | ||
GENESIS_JSON_PATH, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"--genesis-file", | |
GENESIS_JSON_PATH, | |
GENESIS_JSON_PATH, |
why isn't this positional argument?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have an answer to that. It seems like we don't positional args in both swarm and kagami and I thought not to break the "tradition".
I thought specifying the argument explicitly makes it less error prone, don't you agree?
@0x009922 what do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think either way is fine, but I have a feeling that explicit argument would be more consistent overall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a feeling that if you're signing something, it's not optional and should be a positional arg.
CLI apps should have one positional argument, that is the object of operation
#[derive(Clone, Debug, Parser)] | ||
#[command(group = ArgGroup::new("private_key").required(true))] | ||
#[command(group = ArgGroup::new("public_key").required(true))] | ||
pub struct SignArgs { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should not be in this module. We want facility to sign all transactions, not just genesis. Genesis is only a special case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@0x009922 do we put it in kagami crypto
or will we have another command?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
now that I think about it the command should be like this kagami crypto sign file
. It can sign any file. Contents of the file should not be relevant. Whether it's transaction or genesis transaction or query
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At the moment I rely on the fact that the file is indeed genesis file - there are verifications for executor via RawGenesisBlock
struct.
What if I move that logic to crypto and leave it somewhat "as is" and we create a task to extend the functionality somewhere down the road? We might never need it and then we just abandon the idea.
Will it be fine?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mversic, I proposed to move it to kagami genesis
from kagami crypto
deliberately.
It's not just signing of a file. In case of genesis.json
, Kagami needs to know that it is a genesis file, to decoded it from JSON properly, read executor_file
, combine it all into a single genesis transaction, and then produce a signature of it. It is quite different from signing any file, interpreting it as just some blob of bytes.
Therefore, I still think kagami genesis
is a proper place for it, since it is tightly coupled to the genesis.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I expect it to remain essentially the same. Only with addition of chain_id
, and changing of transactions: Instruction[][]
to transaction: Instruction[]
{
"chain_id": "0-0",
"transaction": [
{
"Register": {
"Domain": {
"id": "wonderland",
"logo": null,
"metadata": {
"key": {
"String": "value"
}
}
}
}
}
],
"executor_file": "./executor.wasm"
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
before I comment on this tell me this. Do you think genesis should be given with --submit-genesis
or should it be submitted like any other transaction with iroha_client
? We no longer need to submit genesis at the time of bootstrapping the network. Admin can bootstrap the network and push genesis later to 1 or multiple nodes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let me disambiguate this. We have:
- what
kagami sign
takes. This can have any JSON schema - what
--submit-genesis
takes. This should have the JSON schema ofSignedTransaction
As far as I'm concerned 1. can be whatever. But 2. must be SignedTransaction
or at best TransactionPayload
. Nothing else should be omitted because the node needs all the info to verify the signature. If you omit a field (such as time_to_live_ms
) then we have to, for each omitted field, either:
- provide genesis as another argument together with
--submit-genesis
- we have to agree that every genesis will have some predefined value for this field
option 1. I will not even entertain. Option 2. is too implicit for my taste. This tells me that kagami sign
should best produce full genesis, not just a signature. And this produced genesis should be SCALE encoded blob because it will have inlined executor or some other smart contract
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By specifying time_to_live_ms
do you talk about signed genesis or just any block in general?
One of the issues for us to have a full SignedTransaction
as a parameter is its size - right now even SCALE encoded version is around 1MB.
Also, before that we didn't have a parameter for --signed-genesis
, it was a flag. Do you want to change that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've lost thread of what we are trying to achieve...
Do you think genesis should be given with --submit-genesis or should it be submitted like any other transaction with iroha_client?
If there is no technical need to have the genesis block anymore, then why have it? Let's get rid of it.
190ee3d
to
d3b579e
Compare
Signed-off-by: VAmuzing <amuzik95@gmail.com>
d3b579e
to
98606e0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for torturing with changes requests, but there are still major questionable things here =(
chain_id: &ChainId, | ||
genesis_key_pair: &KeyPair, | ||
) -> GenesisTransaction { | ||
Self(raw_block.unify().sign(chain_id.clone(), genesis_key_pair)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leaves allocation to the caller if they need to:
chain_id: &ChainId, | |
genesis_key_pair: &KeyPair, | |
) -> GenesisTransaction { | |
Self(raw_block.unify().sign(chain_id.clone(), genesis_key_pair)) | |
chain_id: ChainId, | |
genesis_key_pair: &KeyPair, | |
) -> GenesisTransaction { | |
Self(raw_block.unify().sign(chain_id, genesis_key_pair)) |
|
||
/// [`SignedGenesisConfig`] contains data that is used for loading signed genesis from config. | ||
#[derive(Debug, Clone, PartialEq, Eq, Decode, Encode)] | ||
pub struct GenesisSignature { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a small note, iroha_genesis::GenesisSignature
contains name duplication, and may be shortened to iroha_genesis::Signature
. Up to your preference here.
Same applies to GenesisSignatureParseError
, GenesisNetwork
etc.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought that we already have lots of "signatures" and decided to distinguish them like that.
And yeah, we had GenesisNetwork
for a long time so it's kinda similar in that sense, or so I thought.
/// Construct `GenesisSignature` from config | ||
/// # Errors | ||
/// Fails if the | ||
pub fn new_genesis_signature( | ||
raw_block: RawGenesisBlock, | ||
chain_id: &ChainId, | ||
genesis_key_pair: &KeyPair, | ||
) -> GenesisSignature { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- It is not related to
impl GenesisNetwork
# Errors
is irrelevant
Consider moving it to RawGenesisBlock
:
impl RawGenesisBlock {
pub fn sign(&self, chain_id: &ChainId, key_pair: &KeyPair) -> GenesisSignature;
}
@@ -275,7 +430,7 @@ impl<S> RawGenesisBlockBuilder<S> { | |||
|
|||
/// Add instruction to the end of genesis transaction | |||
pub fn append_instruction(mut self, instruction: impl Into<InstructionBox>) -> Self { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess it should be renamed too:
pub fn append_instruction(mut self, instruction: impl Into<InstructionBox>) -> Self { | |
pub fn push_instruction(mut self, instruction: impl Into<InstructionBox>) -> Self { |
Or better keep using append_instruction
everywhere - this will reduce code changes!
algorithm: crypto::AlgorithmArg, | ||
/// Private key (in string format) to sign genesis block | ||
#[clap(long, group = "private_key")] | ||
private_key_string: Option<String>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please replace String
with a type that shows what this string is, which shows how it parses it.
genesis_keypair="--key-pair '{\"public_key\": \"ed01208BA62848CF767D72E7F7F4B9D2D7BA07FEE33760F79ABE5597A51520E292A0CB\",\"private_key\": {\"algorithm\": \"ed25519\",\"payload\": \"8f4c15e5d664da3f13778801d23d4e89b76e94c1b94b389544168b6cb894f84f8ba62848cf767d72e7f7f4b9d2d7ba07fee33760f79abe5597a51520e292a0cb\"}}'" | ||
genesis_signature="--signature 9030303030303030302d303030302d303030302d303030302d3030303030303030303030308c4b28660000000000811b2c0400808ba62848cf767d72e7f7f4b9d2d7ba07fee33760f79abe5597a51520e292a0cb0101819b61a8c82fc6565c5acc7f92ca6dc07ccd39fcc47dee3d6f85d59ec7b52fe651b20b985b401546b49a50057b4311beaccb822b8ef2cf4e25ce3fab3769c70e" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
iroha_swarm
needs to know only the signature
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that iroha_swarm
does not need to know full key-pair, but it does need to know the public key of genesis, isn't it? Otherwise we cannot verify it.
Either provide the genesis by setting `--submit-genesis` argument, `genesis.file`, \ | ||
`genesis.public_key` and `genesis.signature` configuration parameters, or increase the number of trusted peers in \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- The only global change is renaming of
genesis.private_key
togenesis.signature
genesis.public_key
is a required parameter anyway, no need to mention it here IMO.
This will produce a diff minimal to the original:
Either provide the genesis by setting `--submit-genesis` argument, `genesis.file`, \ | |
`genesis.public_key` and `genesis.signature` configuration parameters, or increase the number of trusted peers in \ | |
Either provide the genesis by setting `--submit-genesis` argument, `genesis.signature`, \ | |
and `genesis.file` configuration parameters, or increase the number of trusted peers in \ |
file: PathBuf, | ||
/// Genesis signature config with the data to reconstruct signed genesis block | ||
signature: GenesisSignatureConfig, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
parameters::actual
is exposed to the rest of Iroha. Those places don't really need to know about config parsing details.
signature: GenesisSignatureConfig, | |
signature: iroha_genesis::GenesisSignature, |
@@ -90,6 +91,31 @@ pub struct Network { | |||
pub idle_timeout: Duration, | |||
} | |||
|
|||
/// Wrapper around `iroha_genesis::GenesisSignature` for type-checking | |||
#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Serialize)] | |||
pub struct GenesisSignatureConfig(iroha_genesis::GenesisSignature); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a newtype needed to handle parsing of GenesisSignature
from the user-provided format. It's place more in the parameters::user
IMO.
pub fn into_genesis_signature(self) -> iroha_genesis::GenesisSignature { | ||
self.0 | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pub fn into_genesis_signature(self) -> iroha_genesis::GenesisSignature { | |
self.0 | |
} | |
pub fn into_inner(self) -> iroha_genesis::GenesisSignature { | |
self.0 | |
} |
} | ||
signatures | ||
.verify(&payload) | ||
.map_err(|_| SignedTransactionParsingError::InvalidSignature)?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not disregard underlying error - more info is better. Put it inside of the InvalidSignature
variant.
You can also use thiserror
's #[from]
or #[source]
attributes for that.
/// Get hex string represented genesis_config for ./genesis.json that is tied to hardcoded keys and chain_id | ||
pub fn get_genesis_signature() -> GenesisSignature { | ||
let manifest_dir = Path::new(env!("CARGO_MANIFEST_DIR")); | ||
let genesis = RawGenesisBlock::from_path(manifest_dir.join("../../configs/swarm/genesis.json")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't genesis signature be computed from a genesis block that is generated within tests? I remember that different tests generate it dynamically depending on their needs.
Description
Draft implementation of removing genesis signing, still need to finish removing all the private keys where they're no longer necessary
Added additional option for kagami onto crypto command, now it has both genesis signing and key-pair generation options
Linked issue
Closes #4225
Benefits
Checklist
CONTRIBUTING.md