Build keccak all toghether (#144)

* Add KeccakFConfig & allocation structure def

The KeccakFConfig contains all of the gadget configurations of the
gadgets plus the logic for the allocations of each of the keccak steps
on each of the regions.

This is the first design guideline that seems can fit in with the infra
we have.
Works with #105

* Remove biguint_to_pallas duplicity

* Add aux functions to switch state repr

We need to move from `FieldExt` to `BigUint` Repr in order to execute
KeccaK intermediate steps so that we can allocate all the intermediate
states of the keccak algorithm inside of the circuit.

Therefore we need functions that allow us to swap between both
representations.

* Add `assign_state` placeholders for Pi and Rho Configs

* Add 24-loop state allocation phase in KeccakConfig

* Add state_assign minus mixing stage

* Add configure initial impl for `KeccakConfig`

* Add basic b9 & b13 ROUND_CTANTS allocation

* Change gadgets state allocation to add out_state

We now also allocate the out_state of the gadget when we allocate the
entire witness for the gadget in keccak.

* Merge `next_input` and state assigment to single fn

We can simply do the assigment of the `out_state`, `state` and
`next_input` in a single function reducing the overhead and the
verbosity.

* Change `q_enable` activations to happen in `assign_state`

* Add missing offset increments in KeccakConfig allocation

* Set IotaB9Config Selector as generic Expression

* Set IotaB13 Selector as Expression

* Change AbsorbConfig design and allocation

We now allocate the Absorb as:
- State Row
- Next Mixing Row
- Out State Row

* Move state transformation fns to arith_helpers mod

* Add MixingConfig preliminary design

* Externalize state conversion functions

* Add out_state computation during `assign_state` runtime for B13 & B9

* Add `State` creation function in arith_helpers

* Change AbsorbConfig assigment to compute out_state internally

* Add assign_state_and_mixing_flag_and_rc for IotaB9Config

* Finalize first MixingConfig configure fn

* Change AbsorbConfig to copy_cell strategy

* Add IotaB13Config Cell copy constrains strategy & modify tests

* Update IotaB9Config assigment functions

* Change KeccakF circuit calls to IotaB9 and Mixing configs

* Fix `state_bigint_to_pallas` slice copy lengths

* Add mixing step to KeccakFArith

* test_absorb_gate: Witness input state to get (Cell, Value) tuples.

* Fix range of `state_to_state_bigint`

* IotaB9:_Fix test_flag wrong assignation_err

* iota_b9: Introduce q_last, q_not_last selectors.

These are used to differentiate between gates for the steady state,
and gates for the final round (where an is_mixing flag is witnessed
by the prover).

In the final round, q_last * flag is used as a composite selector.

* Add IotaB9 missing test cases

* IotaB13: Add internal selector + flag setup

With the previous setup, the gate was producing `ConstraintPoisoned` due
to the usage of `round_ctant_b13` at rotation:next to store the
`is_mixing` flag inside.

It also was activated/deactivated following the same bool logic as
IotaB9, and has been changed.

- IotaB13 now activates when `is_mixing = false` so no matter the inputs
  the verification will pass as the gate is not active.
- IotaB13 contains now an internal selector `q_mixing` which is always
  active and prevents the gate equations to fail due to queriyng
  `round_ctant_b13` cells that they shouldn't.

This completes all the development needed for IotaB9 and IotaB13 in
order to add them inside the `MixingConfig` and so work towards closing
issue #105

* Absorb: Add internal selector + flag setup

With the previous setup, the gate was producing `ConstraintPoisoned` due
to the usage of `absorb_next_inputs` at rotation:next to store the
`is_mixing` flag inside.

It also was activated/deactivated following the same bool logic as
IotaB9, and has been changed.

- Absorb now activates when `is_mixing = false` so no matter the inputs
  the verification will pass as the gate is not active.
- Absorb contains now an internal selector `q_mixing` which is always
  active and prevents the gate equations to fail due to queriyng
  `absorb_next_inputs` cells that they shouldn't.

ASSIGNATION MAP:
- STATE (25 columns) (offset -1)
- NEXT_INPUTS (17 columns) + is_mixing flag (1 column) (offset +0) (current rotation)
- OUT_STATE (25 columns) (offset +1)

This completes all the development needed for `AbsorbConfig` in
order to add them inside the `MixingConfig` and so work towards closing
issue #105

* Add state computation fn's for configs

It's much easier, clean and less verbose to compute
`in_state`, `out_state` and `next_inputs` with an associated function
for the MixingConfig sub-configs. And also makes the tests much less
verbose.

* Update StateBigint in compute_states signatures

* Mixing: Add `MixingConfig` impl + tests lacking base conversion

* mixing: Witness flag in state assignation

* Rho: Derive `Debug` for all configs

* xi: Apply copy_constraints for xi inputs

It is critical for the correctness of the keccak circuit to apply copy
constraints between the gates while executing the rounds.

Works towards solving: #219

* Add OFFSET associated consts

* Ignore failing Mixing tests

* Clippy fixes

* Replace pallas by field

* Add zeroed_bytes assertion

Co-authored-by: ying tong <yingtong@z.cash>

keccak256/Cargo.toml

@@ -13,3 +13,6 @@ num-bigint = "0.4.2"
 num-traits = "0.2.14"
 pairing = { git = 'https://github.com/appliedzkp/pairing', package = "pairing_bn256" }
 plotters = { version = "0.3.0", optional = true }
+
+[dev-dependencies]
+pretty_assertions = "1.0"

keccak256/src/arith_helpers.rs

@@ -1,6 +1,9 @@
+use crate::common::State;
 use itertools::Itertools;
 use num_bigint::BigUint;
 use num_traits::{One, Zero};
+use pairing::arithmetic::FieldExt;
+use pairing::bn256::Fr as Fp;
 use std::ops::{Index, IndexMut};
 
 pub const B13: u64 = 13;
@@ -21,7 +24,7 @@ pub type Lane13 = BigUint;
 pub type Lane9 = BigUint;
 
 pub struct StateBigInt {
-    xy: Vec<BigUint>,
+    pub(crate) xy: Vec<BigUint>,
 }
 impl Default for StateBigInt {
     fn default() -> Self {
@@ -33,6 +36,17 @@ impl Default for StateBigInt {
     }
 }
 
+impl From<State> for StateBigInt {
+    fn from(state: State) -> Self {
+        let xy = state
+            .iter()
+            .flatten()
+            .map(|num| BigUint::from(*num))
+            .collect();
+        Self { xy }
+    }
+}
+
 impl StateBigInt {
     pub fn from_state_big_int<F>(a: &StateBigInt, lane_transform: F) -> Self
     where
@@ -183,6 +197,21 @@ pub fn convert_b9_lane_to_b2_normal(x: Lane9) -> u64 {
         .unwrap_or(0)
 }
 
+pub fn big_uint_to_field<F: FieldExt>(a: &BigUint) -> F {
+    let mut b: [u64; 4] = [0; 4];
+    let mut iter = a.iter_u64_digits();
+
+    for i in &mut b {
+        *i = match &iter.next() {
+            Some(x) => *x,
+            None => 0u64,
+        };
+    }
+
+    // Workarround since `FieldExt` does not impl `from_raw`.
+    F::from_bytes(&Fp::from_raw(b).to_bytes()).unwrap()
+}
+
 /// This function allows us to inpect coefficients of big-numbers in different
 /// bases.
 pub fn inspect(x: BigUint, name: &str, base: u64) {
@@ -198,3 +227,58 @@ pub fn inspect(x: BigUint, name: &str, base: u64) {
     }
     println!("inspect {} {} info {:?}", name, x, info);
 }
+
+pub fn state_to_biguint<F: FieldExt>(state: [F; 25]) -> StateBigInt {
+    StateBigInt {
+        xy: state
+            .iter()
+            .map(|elem| elem.to_bytes())
+            .map(|bytes| BigUint::from_bytes_le(&bytes))
+            .collect(),
+    }
+}
+
+pub fn state_to_state_bigint<F: FieldExt, const N: usize>(
+    state: [F; N],
+) -> State {
+    let mut matrix = [[0u64; 5]; 5];
+
+    let mut elems: Vec<u64> = state
+        .iter()
+        .map(|elem| elem.to_bytes())
+        // This is horrible. But FieldExt does not give much better alternatives
+        // and refactoring `State` will be done once the
+        // keccak_all_togheter is done.
+        .map(|bytes| {
+            assert!(bytes[8..32] == vec![0u8; 24]);
+            let mut arr = [0u8; 8];
+            arr.copy_from_slice(&bytes[0..8]);
+            u64::from_le_bytes(arr)
+        })
+        .collect();
+    elems.extend(vec![0u64; 25 - N]);
+    (0..5).into_iter().for_each(|idx| {
+        matrix[idx].copy_from_slice(&elems[5 * idx..(5 * idx + 5)])
+    });
+
+    matrix
+}
+
+pub fn state_bigint_to_field<F: FieldExt, const N: usize>(
+    state: StateBigInt,
+) -> [F; N] {
+    let mut arr = [F::zero(); N];
+    let vector: Vec<F> = state
+        .xy
+        .iter()
+        .map(|elem| {
+            let mut array = [0u8; 32];
+            let bytes = elem.to_bytes_le();
+            array[0..bytes.len()].copy_from_slice(&bytes[0..bytes.len()]);
+            array
+        })
+        .map(|bytes| F::from_bytes(&bytes).unwrap())
+        .collect();
+    arr[0..N].copy_from_slice(&vector[0..N]);
+    arr
+}

keccak256/src/circuit.rs

@@ -0,0 +1,253 @@
+use super::gates::{
+    absorb::{AbsorbConfig, ABSORB_NEXT_INPUTS},
+    iota_b13::IotaB13Config,
+    iota_b9::IotaB9Config,
+    pi::PiConfig,
+    rho::RhoConfig,
+    theta::ThetaConfig,
+    xi::XiConfig,
+};
+use crate::{
+    arith_helpers::*, common::{ROUND_CONSTANTS, PERMUTATION, ROTATION_CONSTANTS}, gates::rho_checks::RhoAdvices,
+};
+use crate::{gates::mixing::MixingConfig, keccak_arith::*};
+use halo2::{
+    circuit::Region,
+    plonk::{Advice, Column, ConstraintSystem, Error, Selector},
+};
+use itertools::Itertools;
+use num_bigint::BigUint;
+use pasta_curves::arithmetic::FieldExt;
+use std::{convert::TryInto, marker::PhantomData};
+
+#[derive(Clone, Debug)]
+pub struct KeccakFConfig<F: FieldExt> {
+    theta_config: ThetaConfig<F>,
+    rho_config: RhoConfig<F>,
+    pi_config: PiConfig<F>,
+    xi_config: XiConfig<F>,
+    iota_b9_config: IotaB9Config<F>,
+    mixing_config: MixingConfig<F>,
+    state: [Column<Advice>; 25],
+    _marker: PhantomData<F>,
+}
+
+impl<F: FieldExt> KeccakFConfig<F> {
+    const B9_ROW: usize = 0;
+    const B13_ROW: usize = 1;
+
+    // We assume state is recieved in base-9.
+    pub fn configure(meta: &mut ConstraintSystem<F>) -> KeccakFConfig<F> {
+        let state = (0..25)
+            .map(|_| {
+                let column = meta.advice_column();
+                meta.enable_equality(column.into());
+                column
+            })
+            .collect_vec()
+            .try_into()
+            .unwrap();
+
+        // theta
+        let theta_config = ThetaConfig::configure(meta.selector(), meta, state);
+        // rho
+        let rho_config = {
+            let cols: [Column<Advice>; 7] = state[0..7].try_into().unwrap();
+            let adv = RhoAdvices::from(cols);
+            let axiliary = [state[8], state[9]];
+
+            let base13_to_9 = [
+                meta.fixed_column(),
+                meta.fixed_column(),
+                meta.fixed_column(),
+            ];
+            let special = [meta.fixed_column(), meta.fixed_column()];
+            RhoConfig::configure(
+                meta,
+                state,
+                &adv,
+                axiliary,
+                base13_to_9,
+                special,
+            )
+        };
+        // Pi
+        let pi_config = PiConfig::configure(meta.selector(), meta, state);
+        // xi
+        let xi_config = XiConfig::configure(meta.selector(), meta, state);
+
+        // Iotab9
+        // Generate advice and instance column for Round constants in base9
+        let round_ctant_b9 = meta.advice_column();
+        let round_constants_b9 = meta.instance_column();
+        let iota_b9_config = IotaB9Config::configure(
+            meta,
+            state,
+            round_ctant_b9,
+            round_constants_b9,
+        );
+        let not_mixing_b9_to_13 = StateConversion::configure(meta);
+
+
+
+        let mixing_config = MixingConfig::configure(meta, state);
+        // in side mixing  let b9_to_13 = StateConversion::configure(meta);
+
+
+        KeccakFConfig {
+            theta_config,
+            rho_config,
+            pi_config,
+            xi_config,
+            iota_b9_config,
+            mixing_config,
+            state,
+            _marker: PhantomData,
+        }
+    }
+
+    pub fn assign_all(
+        &self,
+        region: &mut Region<'_, F>,
+        mut offset: usize,
+        state: [F; 25],
+        out_state: [F;25],
+        flag: bool,
+        next_mixing: Option<[F; ABSORB_NEXT_INPUTS]>,
+        absolute_row_b9: usize,
+        absolute_row_b13: usize,
+    ) -> Result<[F; 25], Error> {
+        // In case is needed
+        let mut state = state;
+
+        // First 23 rounds
+        for round in 0..PERMUTATION {
+            // State in base-13
+            // theta
+            state = {
+                // Apply theta outside circuit
+                let out_state = KeccakFArith::theta(&state_to_biguint(state));
+                let out_state = state_bigint_to_pallas(out_state);
+                // assignment
+                self.theta_config
+                    .assign_state(region, offset, state, out_state)?
+            };
+
+            offset += ThetaConfig::OFFSET;
+
+            // rho
+            state = {
+                // Apply rho outside circuit
+                let out_state = KeccakFArith::rho(&state_to_biguint(state));
+                let out_state = state_bigint_to_pallas(out_state);
+                // assignment
+                self.rho_config
+                    .assign_region(region, offset, out_state)?;
+                out_state
+            };
+            // Outputs in base-9 which is what Pi requires.
+            offset += RhoConfig::OFFSET;
+
+            // pi
+            state = {
+                // Apply pi outside circuit
+                let out_state = KeccakFArith::pi(&state_to_biguint(state));
+                let out_state = state_bigint_to_pallas(out_state);
+                // assignment
+                self.pi_config
+                    .assign_state(region, offset, state, out_state)?
+            };
+
+            offset += PiConfig::OFFSET;
+
+            // xi
+            state = {
+                // Apply xi outside circuit
+                let out_state = KeccakFArith::xi(&state_to_biguint(state));
+                let out_state = state_bigint_to_pallas(out_state);
+                // assignment
+                self.xi_config
+                    .assign_state(region, offset, state, out_state)?
+            };
+
+            offset += XiConfig::OFFSET;
+
+            // iota_b9
+            state = {
+            let out_state = KeccakFArith::iota_b9(&state_to_biguint(state), ROUND_CONSTANTS[round]);
+            let out_state = state_bigint_to_pallas(out_state); 
+            self
+                .iota_b9_config
+                .not_last_round(region, offset, state, out_state, round)?;
+                out_state
+            };
+            offset += IotaB9Config::OFFSET;
+            // The resulting state is in Base-13 now. Which is what Theta
+            // requires again at the start of the loop.
+
+            self.not_mixing_b9_to_13.not_last_round();
+        }
+
+        // Final round.
+        let round = PERMUTATION;
+        // PERMUTATION'th round
+        // State in base-13
+        // theta
+        state = {
+            // Apply theta outside circuit
+            let out_state = KeccakFArith::theta(&state_to_biguint(state));
+            let out_state = state_bigint_to_pallas(out_state);
+            // assignment
+            self.theta_config
+                .assign_state(region, offset, state, out_state)?
+        };
+
+        offset += 1;
+
+        // rho
+        state = {
+            // Apply rho outside circuit
+            let out_state = KeccakFArith::rho(&state_to_biguint(state));
+            let out_state = state_bigint_to_pallas(out_state);
+            // assignment
+            self.rho_config
+                .assign_state(region, offset, state, out_state)?
+        };
+        // Outputs in base-9 which is what Pi requires.
+        offset += 1;
+
+        // pi
+        state = {
+            // Apply pi outside circuit
+            let out_state = KeccakFArith::pi(&state_to_biguint(state));
+            let out_state = state_bigint_to_pallas(out_state);
+            // assignment
+            self.pi_config
+                .assign_state(region, offset, state, out_state)?
+        };
+
+        offset += PiConfig::OFFSET;
+
+        // xi
+        state = {
+            // Apply xi outside circuit
+            let out_state = KeccakFArith::xi(&state_to_biguint(state));
+            let out_state = state_bigint_to_pallas(out_state);
+            // assignment
+            self.xi_config
+                .assign_state(region, offset, state, out_state)?
+        };
+
+        offset += XiConfig::OFFSET;
+
+        // Mixing step
+        state = {
+            let out_state = KeccakFArith::mixing(&state_to_biguint(state), next_mixing, ROUND_CONSTANTS[round]);
+            let out_state = state_bigint_to_pallas(out_state);
+            self.mixing_config.assign_state(region, offset, state, out_state, flag, next_mixing, round, round)?;
+            out_state
+        };
+
+        Ok(state)
+    }
+}

keccak256/src/gates.rs

@@ -1,7 +1,11 @@
+#![allow(dead_code)]
+#![allow(clippy::type_complexity)]
+#![allow(clippy::too_many_arguments)]
 pub mod absorb;
 pub mod gate_helpers;
 pub mod iota_b13;
 pub mod iota_b9;
+pub mod mixing;
 pub mod pi;
 pub mod rho;
 pub mod rho_checks;