[ckb_chain]: Calculate tx fees when switch is Switch::DISABLE_ALL

[eval-exec]

What problem does this PR solve?

Problem Summary:

ckb/chain/src/chain.rs

Lines 804 to 810 in b78864d

	} else {
	txn.attach_block(b)?;
	attach_block_cell(&txn, b)?;
	mmr.push(b.digest())
	.map_err(|e| InternalErrorKind::MMR.other(e))?;
	self.insert_ok_ext(&txn, &b.header().hash(), ext.clone(), None, None)?;
	}

In fn reconcile_main_chain, if switch is Switch::DISABLE_ALL, tx fees won't be calculated and saved into BlockExt, this will cause RewardCalculator::block_reward_internal failed to calculate right block_reward:

ckb/util/reward-calculator/src/lib.rs

Lines 95 to 102 in 3aa3051

	let txs_fees = self.txs_fees(target)?;
	let proposal_reward = self.proposal_reward(parent, target)?;
	let (primary, secondary) = self.base_block_reward(target)?;
	let total = txs_fees
	.safe_add(proposal_reward)?
	.safe_add(primary)?
	.safe_add(secondary)?;

What is changed and how it works?

What's Changed:

• Make FeeCalculator and it's new and transaction_fee methods public.
• Calculate tx fees and txs_sizes, then insert tx fees and tx_sizes to BlockExt when switch is Switch::DISABLE_ALL

TODO

• Add unit test, start a mock chain and process_block in Switch::DISABLE_ALL mode, then continue to process_block in Switch::NONE mode.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code ci-runs-only: [ quick_checks,linters ]

Side effects

• None

Release note

None: Exclude this PR from the release note.

[eval-exec]

ckb/util/reward-calculator/src/lib.rs

Lines 150 to 254 in 3aa3051

	/// Earliest proposer get 40% of tx fee as reward when tx committed
	/// block H(19) target H(13) ProposalWindow(2, 5)
	/// target current
	/// / /
	/// 10 11 12 13 14 15 16 17 18 19
	/// \ \ \ \______/___/___/___/
	/// \ \ \________/___/___/
	/// \ \__________/___/
	/// \____________/
	///
	fn proposal_reward(
	&self,
	parent: &HeaderView,
	target: &HeaderView,
	) -> CapacityResult<Capacity> {
	let mut target_proposals = self.get_proposal_ids_by_hash(&target.hash());
	let proposal_window = self.consensus.tx_proposal_window();
	let proposer_ratio = self.consensus.proposer_reward_ratio();
	let block_number = parent.number() + 1;
	let store = self.store;
	let mut reward = Capacity::zero();
	// Transaction can be committed at height H(c): H(c) > H(w_close)
	let competing_commit_start = cmp::max(
	block_number.saturating_sub(proposal_window.length()),
	1 + proposal_window.closest(),
	);
	let mut proposed: HashSet<ProposalShortId> = HashSet::new();
	let mut index = parent.to_owned();
	// NOTE: We have to ensure that `committed_idx_proc` and `txs_fees_proc` return in the
	// same order, the order of transactions in block.
	let committed_idx_proc = |hash: &Byte32| -> Vec<ProposalShortId> {
	store
	.get_block_txs_hashes(hash)
	.into_iter()
	.skip(1)
	.map(|tx_hash| ProposalShortId::from_tx_hash(&tx_hash))
	.collect()
	};
	let txs_fees_proc = |hash: &Byte32| -> Vec<Capacity> {
	store
	.get_block_ext(hash)
	.expect("block ext stored")
	.txs_fees
	};
	let committed_idx = committed_idx_proc(&index.hash());
	let has_committed = target_proposals
	.intersection(&committed_idx.iter().cloned().collect::<HashSet<_>>())
	.next()
	.is_some();
	if has_committed {
	for (id, tx_fee) in committed_idx
	.into_iter()
	.zip(txs_fees_proc(&index.hash()).iter())
	{
	// target block is the earliest block with effective proposals for the parent block
	if target_proposals.remove(&id) {
	reward = reward.safe_add(tx_fee.safe_mul_ratio(proposer_ratio)?)?;
	}
	}
	}
	while index.number() > competing_commit_start && !target_proposals.is_empty() {
	index = store
	.get_block_header(&index.data().raw().parent_hash())
	.expect("header stored");
	// Transaction can be proposed at height H(p): H(p) > H(0)
	let competing_proposal_start =
	cmp::max(index.number().saturating_sub(proposal_window.farthest()), 1);
	let previous_ids = store
	.get_block_hash(competing_proposal_start)
	.map(|hash| self.get_proposal_ids_by_hash(&hash))
	.expect("finalize target exist");
	proposed.extend(previous_ids);
	let committed_idx = committed_idx_proc(&index.hash());
	let has_committed = target_proposals
	.intersection(&committed_idx.iter().cloned().collect::<HashSet<_>>())
	.next()
	.is_some();
	if has_committed {
	for (id, tx_fee) in committed_idx
	.into_iter()
	.zip(txs_fees_proc(&index.hash()).iter())
	{
	if target_proposals.remove(&id) && !proposed.contains(&id) {
	reward = reward.safe_add(tx_fee.safe_mul_ratio(proposer_ratio)?)?;
	}
	}
	}
	}
	Ok(reward)
	}

In there:

ckb/util/reward-calculator/src/lib.rs

Lines 225 to 227 in 3aa3051

	// Transaction can be proposed at height H(p): H(p) > H(0)
	let competing_proposal_start =
	cmp::max(index.number().saturating_sub(proposal_window.farthest()), 1);

I think when parent.number() <= 11, competing_proposal_start will be always be 1, then proposed will always contains the tx0 which proposed in block 1 and committed in block 3.

Then !proposed.contains(&id) will be always false, so block 12 doesn't calculate proposal reward for tx0 which proposed in block 1. https://github.com/nervosnetwork/ckb/blob/3aa30517f31b798e25c955e053b497c24c2f64d0/util/reward-calculator/src/lib.rs#L247-L249C22

---

A workaround way to avoid this case is "generate many blocks (blocks count >=13)" before gen_block_with_proposal_txs_and_commit_txs in this unit test.

---

Confirmed: For the miners who mined blocks [1-11], the proposal reward doesn't be issued.

[eval-exec]

It's weired, all of the CI jobs are success on this PR, but failed on merge queue:
https://github.com/nervosnetwork/ckb/actions/runs/4628209279/jobs/8187022815
https://github.com/nervosnetwork/ckb/actions/runs/4628209282/jobs/8187021573

Investigating why...

Update: rebased my branch on develop.

Figured out, unit tests and integration tests are skipped on "push" action CI, they only are triggred on "merge queue" action CI, after this PR: #3837

I need to fix all unit tests.

[eval-exec]

❯ cat ci_unit_tests_ubuntu/7_Run\ if\ \[\[\ run\ ==\ run\ \]\]\ \&\&\ \[\[\ run\ ==\ run\ \]\]\;then.txt | grep STDERR
2023-04-06T11:20:05.0316576Z --- STDERR:              ckb-chain tests::block_assembler::test_candidate_uncles_retain ---
2023-04-06T11:20:05.0390549Z --- STDERR:              ckb-chain tests::block_assembler::test_block_template_timestamp ---
2023-04-06T11:20:05.5844601Z --- STDERR:              ckb-chain tests::block_assembler::test_package_basic ---
2023-04-06T11:20:05.7267437Z --- STDERR:              ckb-chain tests::block_assembler::test_package_low_fee_descendants ---
2023-04-06T11:20:09.4716241Z --- STDERR:              ckb-chain tests::block_assembler::test_block_template_timestamp ---
2023-04-06T11:20:09.4754594Z --- STDERR:              ckb-chain tests::block_assembler::test_candidate_uncles_retain ---
2023-04-06T11:20:09.4793048Z --- STDERR:              ckb-chain tests::block_assembler::test_package_basic ---
2023-04-06T11:20:09.4852449Z --- STDERR:              ckb-chain tests::block_assembler::test_package_low_fee_descendants ---

These unit tests are failed on merge queue CI jobs.