Solidity packing doesn't support arrays

[raineorshine]

I have been trying to reproduce Solidity's sha3 in Javascript, and have gotten everything to work with my own code except arrays. Once I found out about this library, I tried to use it for arrays, but I haven't been able to get the same hash as Solidity produces yet. It would be great if you might be able to help me understand this better.

How would I generate the same sha3 hash using ethereumjs-abi as in this Solidity example?

contract Contract {
    function Test() constant returns(bytes32) {
        uint[] memory amounts = new uint[](2);
        amounts[0] = 1;
        amounts[1] = 2;

        bytes8[] memory tickers = new bytes8[](2);
        tickers[0] = "BTC";
        tickers[1] = "LTC";

        // 0x4282aea708b799612aa9d311309ef4b8eb1374cf0740c3e9d5914cb5ce9b93f2
        return sha3(amounts, tickers);
    }
}

My attempt gives an error of Cannot read property 1 of undefined:

abi.soliditySHA3(['uint[]', 'bytes8[]'], [[1,2], ['BTC', 'LTC']])

[axic]

There was a bug in using the shorthand-types (i.e. uint) in master a while ago. Yet to release it.

Either use the master or change uint to uint256.

[raineorshine]

> soliditySHA3(['uint256[]', 'bytes8[]'], [[1,2], ['BTC', 'LTC']])
TypeError: Cannot read property '1' of null
    at parseTypeN (/Users/raine/projects/tags/node_modules/ethereumjs-abi/lib/index.js:42:42)
    at Function.ABI.solidityPack (/Users/raine/projects/tags/node_modules/ethereumjs-abi/lib/index.js:454:14)
    at ABI.soliditySHA3 (/Users/raine/projects/tags/node_modules/ethereumjs-abi/lib/index.js:487:25)
    at repl:1:1
    at REPLServer.defaultEval (repl.js:252:27)
    at bound (domain.js:287:14)
    at REPLServer.runBound [as eval] (domain.js:300:12)
    at REPLServer.<anonymous> (repl.js:417:12)
    at emitOne (events.js:95:20)
    at REPLServer.emit (events.js:182:7)

[axic]

@raineorshine you are right it doesn't support arrays.

I would like to rework the library and make the types ES6 classes. That should simplify both the encoding/decoding and the solidity packing. Are you interested in helping out?

[raineorshine]

@axic Very interested, but definitely too busy the next couple weeks as I finish up a project and prep for Devcon2. Watching the thread though! Might have time next month.

[djrtwo]

@axic @raineorshine I'm working on implementing a soliditySha3 function in python but can't figure out how the values are packed for arrays. Do either of you know/have a link to documentation?

Thanks

[raineorshine]

@djrtwo Maybe here? https://github.com/brix/crypto-js/blob/develop/src/sha3.js

Not sure; I know it was added to web3-utils.

[djrtwo]

Solidity packs values from arrays as size 256 regardless of type.

So in Solidity:
uint8[] uint8Array = [8, 9]; keccak256(uint8Array);
is equivalent to:
keccak256(hex'00000000000000000000000000000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000000000000009');
rather than what i expected:
keccak256(hex'0809');

Same goes for arrays of all other fixed types. 160bit addresses are padded to 256bits, Booleans (0x01, 0x00) are padded to 256 bits.

I might get arrange to implementing this in this library sometime, but if someone gets to it first, I wanted y'all to know! Took me tracing a Solidity call to figure it out...

Cheers!

[holgerd77]

Not sure, is this still an issue?

[raineorshine]

Is there a test for it? If not, I'd guess it still exists.

[tommyz7]

For the time being (as @djrtwo explained) you can do the following:

contract GenHash {
    function getHash(string title, uint256 num, address[] addr, uint256 num2) public pure returns (bytes32) {
        return keccak256(abi.encodePacked(title, num, addr, num2));
    }
}

and in JS..

function getHash(title, num, arr, num2) {
  let args = []
  arr.map((addr, index) => {
    args[index] = {t: 'bytes', v: web3.utils.leftPad(addr, 64)}
  })
  return web3.utils.soliditySha3(
    {t: 'string', v: title},
    {t: 'uint256', v: num},
    ...args,
    {t: 'uint256', v: num2}
  );
}

[danjm]

There appear to be two PRs which attempt to resolve this issue #47 and #50. The algorithms used in each are fairly different. I will review each and run each against a more robust set of test cases, then move forward the most appropriate PR.

[holgerd77]

@danjm sounds good!

[forshtat]

@danjm @holgerd77 Hello. Any progress on this one, by chance? I mean, this issue is quite frustrating and prevents us from using the released version of the module, and also, just saying, this is very basic functionality that has been reported missing since the year 2016, and I see that two pull requests with fixes were already made. Do you need any assistance with this, maybe? Will be glad to help. Thanks!

[holgerd77]

@alex-forshtat-tbk Sorry, we just have too many libraries to maintain with too few people. Could you eventually give some short comparison of the two PRs open and give some opinion which one you prefer?

[forshtat]

@holgerd77 Without diving too deep, in my opinion, the PR #47 extracts a nice piece of solidity encoding logic into a 'solidityHexValue' function that encapsulates a hex representation of solidity types and has a cute recursive special condition for arrays, that guarantees multi-dimensional array support. Also, it passes the tests.

In comparison, #50 is a complete mess. "toSolidityBytes32" is added, but not used by the commit itself, and the code basically duplicates everything again inside 'isArray' branch for no reason. Multi-dimensional arrays are not supported.

[holgerd77]

@alex-forshtat-tbk thanks, very helpful! I will try to use the "Update branch" function from GitHub and manually resolve conflicts in the next couple of days, since the original author of the PR is not responsive anymore (which is no wonder 😛).